search

codalab / codalab-worksheets

A collaborative platform for reproducible research (web interface and CLI).
Other
157 stars 84 forks source link

Sometimes the worker manager's deployed workers use a stale image. #4407

Open AndrewJGaut opened 1 year ago

AndrewJGaut commented 1 year ago

Sometimes workers deployed by the Azure batch worker manager deploy a stale image (i.e. an older image before recent changes were made to the worker).

However, when I docker pulled the image with the same name and tag, I got the updated image. So I'm not sure what's happening. Likely, it has to do with the way Azure is pulling the image.

The image name is set by this line: worker_image: str = 'codalab/worker:' + os.environ.get('CODALAB_VERSION', 'latest') and then it's used in the TaskContainerSettings here:

container_settings=TaskContainerSettings(
                image_name=worker_image, container_run_options=' '.join(task_container_run_options)
            ),
percyliang commented 1 year ago

Are there instructions to reproduce this bug?