Florence finetunes in workflows

[probicheaux]

Description

A few notable changes to florence models:

• Adds in a new (fake) "task_type" unstructured, to allow the user to type whatever they want as a prompt
• Creates a v2 block that remaps model_version to model_id so that the user can search their finetuned florence 2 models in app
• tweaks list of required model files to avoid always redownloading florence2 models
• adds no_repeat_ngram_size=0 to our transformers generation args

Why we set no_repeat_ngram_size=0:

I found a very sneaky bug destroying the ability of the model to generate valid json, where this parameter was set to 3. Having the parameter set to 3 means that once the model has seen a sequence of 3 tokens, it can never repeat that sequence.

Here's the generation for a receipt json before and after making this change: Generation before:

{'<RECEIPT>': '{"subtotal": "$27.36", "total":{"29.96"}'}

Generation after:

{'<RECEIPT>': '{"subtotal": "$27.36", "total": "$29.96"}'}

Here's why -- here's the tokens the model generated after I fixed the parameter.

['</s>', '<s>', '{"', 'sub', 'total', '":', ' "$', '27', '.', '36', '",', ' "', 'total', '":', ' "$', '29', '.', '96', '"}', '</s>']

Notice the repeated sequence of tokens:

'total', '":', ' "$',

that the model was unable to generate before this change.

Type of change

Please delete options that are not relevant.

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

tested and deployed thouroughly on localhost

Any specific deployment considerations

no

Docs

• [ ] Docs updated? What were the changes:

[probicheaux]

@PawelPeczek-Roboflow good to go?