search

cyan2k / molmo-7b-bnb-4bit

4bit bitsandbytes quants of the best 7B vlms
Apache License 2.0
21 stars 3 forks source link

modified inference script and custom preprocessing script #1

Closed BBC-Esq closed 1 month ago

BBC-Esq commented 1 month ago

Added two files:

1) A modified preprocessing script that removes the complex tensorflow dependencies and instead relies on torch and torchvision, which are already dependencies anyways...so why use tensorflow at all?

2) An inference script that works assuming you use the modified image_preprocessing_molmo.py script provided above. NOTE: this script differs from the original inference script also in that it takes a hardcoded path to the molmo repository files instead of relying on the repo id and automatically downloading them. Feel free to modify for your use-case, however.

BBC-Esq commented 1 month ago

Can you review this?

cyan2k commented 1 month ago

done and thanks!

Nice work with the tensorflow free preprocessing! Had this on my to do as well but didn't found the time so far. Didn't even had time to review PRs ;)

BBC-Esq commented 1 month ago

You're welcome. BTW, any chance I can get the script or the command that was used to create the model and associated .json files that were modified? I'd like to experiment with different conversion parameters and what not.