Coinbase has conducted a test to see ChatGPT’s utility in conducting security analysis for ERC-20 tokens. While it didn’t hit the desired accuracy, Coinbase notes that it shows potential.
Coinbase is showing more interest in the application of ChatGPT in the crypto industry, with the exchange conducting a test using the AI application. The San Francisco-based exchange conducted a test for ERC-20 token security analysis, publishing the results in a blog post on March 20.
The test focused on performing automated token reviews that utilized Coinbase’s own ERC-20 token review framework. Coinbase notes that “ChatGPT did not hit the accuracy bar to clearly demonstrate that it should be integrated into our asset review process,” though the exchange states that it did show potential.
Coinbase compared the results of ChatGPT’s analysis with those of its internal team, which has internal tools to review a blockchain project. The company compared a total of 20 smart contract risk scores between ChatGPT and manual reviews, with ChatGPT getting the same result 60% of the time. However, in the eight incorrect outputs, the AI labeled a high-risk asset as a low-risk one five out of ten times.
Regarding specific concerns, Coinbase stated that “ChatGPT is not capable of recognizing when it lacks context to perform a robust security analysis.” Other concerns were that the AI is prone to inconsistency and the fact upgrades to ChatGPT may result in output instability.
Coinbase Executive Tested the AI Earlier
This isn’t the first time that Coinbase has experimented with ChatGPT. Coinbase’s Conor Grogan stated earlier this month that ChatGPT was successful in finding bugs in smart contracts. It’s one of the most significant and practical uses of AI in the crypto industry yet.
Grogan offered ChatGPT a live Ethereum contract, asking it to discover any security vulnerabilities. Not only did it discover exploits, but it also offered a specific means to exploit the contract.
However, skeptics state that the AI model could have simply searched the internet for the information, as the contract was from 2018. It may have trouble with new unsecured contracts that haven’t been analyzed.
ChatGPT Continues To Be the Talk of the Town
ChatGPT continues to be the darling of the tech world, with the newly released GPT-4 model setting off another wave of hype. It can now process images, which opens up a number of possibilities for users. Of course, such rapid improvement has only increased the concerns related to the effects it could have on the job market.
However, not everyone seems worried about the impact that ChatGPT will do on jobs. Developers of Ethereum, speaking at an event in Dubai, don’t believe that the application will replace developers.
ChatGPT mania has set off a rally in AI-based tokens in the crypto market. However, some are scams, so users must exercise caution in the market.
Disclaimer
BeInCrypto has reached out to company or individual involved in the story to get an official statement about the recent developments, but it has yet to hear back.
