search

GeniusVentures / thirdparty

2 stars 2 forks source link

Integrate IPFS loading from ipfs-lite-cpp library to load File formats for MNN #14

Open SuperGNUS opened 1 year ago

SuperGNUS commented 1 year ago

After integrating MNN into thirdparty as a submodule, remove Kompute from the build and then integrate the loading through ipfs-lite-cpp to load the file formats MNN supports. I think there is the TensorFlow file format and others. Should be a wrapper around their file system loading code.

scorpioluck20 commented 1 year ago

I'm working on this issue. BRs, Tan Nguyen

scorpioluck20 commented 1 year ago

Implement was done for Linux OS. ipfs-lite-cpp: https://github.com/GeniusVentures/ipfs-lite-cpp/pull/1 thirdparty: https://github.com/GeniusVentures/thirdparty/pull/23

SuperGNUS commented 1 year ago

Please refactor this. There is no reason this needs to be embedded in ipfs-lite-cpp, it should be its own library and when added to SuperGenius, it should hook itself into ipfs-lite-cpp, not be embedded in the library

scorpioluck20 commented 1 year ago

Update design description:

Please don't hack in other features to libraries where they don't belong.  
The MNN parser has no business being a subdirectory of ipfs-lite-cpp.  
I just updated the feature request, so refactor it into it own repo called something like MNNParser.  
Then SuperGenius can call MNNParser, when it receives a .MNN file or whatever the suffix is.

If the MNNParser is also meant to be a streaming data handler for the ML/AI data, 
please make it separate and when constructing the main MNN class,
have it hook into the streaming part of ipfs-lite-cpp.
scorpioluck20 commented 1 year ago

https://github.com/GeniusVentures/MNNLoader| MNN Loader library under development. Will be integrate it to ipfs-lite-cpp when it done.

scorpioluck20 commented 1 year ago

PR was create: #32 Build is working on Linux and Windows OS. For other OS build got failed by another library. For the SuperGenius can call MNNLoader library to loading or print-out information .MNN file. Just follow-up with the MNNExample program include in MNNLoader. We can closed this ticker after merge the PR #32 BRs, TN

itsafuu commented 7 months ago

MNN expects TensorFlow, etc to be converted to mnn format using the converter util. We should probably just have this done before it is sent out to clients for processing. Otherwise, the ASyncIOmanager can load any type of file from IPFS as is, so I think this is fine.

Super-Genius commented 7 months ago

Just a comment to see if github notifications go to element. Maybe move this do done @itsafuu