try to release `lead` model in huggingface.io under our org `Ben the bridge engine`

[larrycai]

after #73 , it is time to try real models in huggingface, lead is one example

learn how to play models with huggingface

• copy lead model related codes to ben-lead-model (repo in huggingface or github)
• release the model into https://huggingface.co/AIBridgeEngine/
• write guideline how to use model remotely. (later can be download in jupyterbook or ben engine), into src/examples
• could be one feature to switch models in UI for easy testing. (load several models from huggingface)

[ThorvaldAagaard]

I have uploaded the single dummy model, as that is stateless, and can be interesting to use for other projects.

So we will have to create an API for using the model.

In SingleDummyEstimates.ipynb there is an example of how to use the model from the local file system

[larrycai]

i am new to it, it is a great chance for me to learn both (model and huggingface)

[ThorvaldAagaard]

Give it a try - you have free hands.

[larrycai]

Checking this in https://github.com/lorserker/ben/blob/main/src/examples/OpeningLead.ipynb

models = Models.from_conf(conf.load('./config/default.conf'),'..')   # loading neural networks
sampler = Sample.from_conf(conf.load('./config/default.conf'), False)  # Load sampling strategies

And can be changed to hf download using https://huggingface.co/docs/huggingface_hub/en/guides/download

import huggingface_hub
import os
lead_id="AIBridgeEngine/Ben-3B-Lead-v0.1"
bidding_id="AIBridgeEngine/Ben-3B-Bidding-v0.1"
biddinginfo_id="AIBridgeEngine/Ben-3B-Biddinginformation-v0.1"
singledummy_id="AIBridgeEngine/Ben-3B-Singledummy-v0.1"
folder="hfmodels"
huggingface_hub.snapshot_download(repo_id=lead_id, local_dir=folder)
huggingface_hub.snapshot_download(repo_id=bidding_id, local_dir=folder)
huggingface_hub.snapshot_download(repo_id=biddinginfo_id, local_dir=folder)
huggingface_hub.snapshot_download(repo_id=singledummy_id, local_dir=folder)

Now as @ThorvaldAagaard suggested

from nn.leader import Leader
from nn.bid_info import BidInfo
from nn.bidder import Bidder
from nn.lead_singledummy import LeadSingleDummy
lead = Leader(f'{folder}/lead-1000000')
biddinginfo = BidInfo(f'{folder}/binfo-100000')
bidder = Bidder("", f'{folder}/bidding-100000')
sd_model = LeadSingleDummy(f'{folder}/lr3-1000000')
models = Models(bidder, biddinginfo, lead, sd_model, None, 0.1, 0.05)
sampler = Sample(0.01, 0.1, 0.03, 0.05, 64, 5000, 5000, 128, 100, False)

we may add OpeningLead_hf.ipynb

Then more ideas ;-), if u think they are good ideas, register issue.

• add redeal https://github.com/anntzer/redeal to have better samples format
• release ben as pip library, so it can be easily loaded into jupyter notebook without this repo (!pip install ben)

Later one PR for this file

[ThorvaldAagaard]

My first thought is that I would like to keep all the model names/configuration in the configuration file.

So perhaps it is a good idea to start by letting hf read the conf-file and download the models specified. Next step would include that the download could check if the model already is present (cached)

We could also just add a parameter in the configuration file, and then download the models in the Models and Sampler classes.

I will add the last 4 models.

In hf we should think about how to name the models, so it is easier to find - perhaps bidding system etc

I almost never use the notebook, but can see the idea of being able to create scripts outside the repo.

Redeal is not important for now

[larrycai]

notebook is good for user/data engineer to analysis the model without knowing other script.

notebook can serve as simple article as well.

[larrycai]

I close this issue as we have something in hf and basic part works.