MorpheusAIs / Morpheus

Morpheus - A Network For Powering Smart Agents - Compute + Code + Capital + Community
https://mor.org/
MIT License
182 stars 142 forks source link

LLM Transaction Response Validator #606

Closed a1t3g0 closed 6 months ago

a1t3g0 commented 6 months ago

The current Metamask model logic assumes that the LLM JSON response for transacting is correct.

Data validation is necessary for any LLM response to ensure (in this case) the JSON returned is correct and valid. This is very important as:

  1. Base-model training is based on a corpus at a set date i.e. they may not have seen the latest updates to Metamask docs (or other API specs) and could return methods, format etc. that is no longer supported
  2. Base-model hallucinations can cause a high confidence response of incorrect data.
  3. API errors will be harder to detect if we’re not certain the correct data was sent
  4. Poor UX (user experience) is likely if we don’t manage the data information flow

Initial proposed solution builds on existing logic to minimise refactoring the code base: These include:

Note: Class ethSendTransactions has been replaced with TransactionClass in order to support several Metamask transaction methods

Other improvements will be added in next PR - in consultation with the repository lead

DavidAJohnston commented 6 months ago

Looks good to me. @betterbrand thoughts?

IshShogun commented 6 months ago

Looks good to me. @betterbrand thoughts?

Looks great to me 😄

betterbrand commented 6 months ago

Looks good to me. I'd like Rakshak to review

DavidAJohnston commented 6 months ago

Moving this over to the new Docs repo.

DavidAJohnston commented 6 months ago

Closing this PR. Please move over to the new repo.