explosion / weasel

🦦 weasel: A small and easy workflow system
MIT License
63 stars 8 forks source link

weasel push: http support for mode 'wb' not implemented #76

Closed justmars closed 8 months ago

justmars commented 8 months ago

Recently tried experimenting with weasel and so far so good save for the last part of saving trained models.

Ran weasel push in two attempts: one endpoint url at R2 and another in S3 (they share some common apis) but getting the following error.

NotImplementedError: http support for mode 'wb' not implemented

The necessary keys (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) are declared in .env and I have the following partial project.yml config:

...
commands:
  - name: "train"
    script:
    - prodigy train x x x
    deps:
    - config.cfg
    outputs:
    - training/${vars.name}/model-best

remotes:
  default: 'https://fullendpoint.url.here/bucket_name'

Not sure if it's faulty config on my part or if I missed something in the docs.

justmars commented 8 months ago

Got it to work, in case helpful to someone:

  1. default: s3://bucket us convention
  2. export ENV vars to virtual env
  3. weasel push