Update 05-run-jobs.Rmd for DCP2

cytomining / profiling-handbook

Image-based Profiling Handbook

https://cytomining.github.io/profiling-handbook/

Creative Commons Zero v1.0 Universal

8 stars 7 forks source link

Update 05-run-jobs.Rmd for DCP2 #57

Closed shntnu closed 3 years ago

shntnu commented 3 years ago

Traceback (most recent call last):
  File "run.py", line 597, in <module>
    setup()
  File "run.py", line 347, in setup
    update_ecs_task_definition(ecs, ECS_TASK_NAME, AWS_PROFILE)
  File "run.py", line 138, in update_ecs_task_definition
    task_definition = generate_task_definition(AWS_PROFILE)
  File "run.py", line 120, in generate_task_definition
    "value": str(MIN_FILE_SIZE_BYTES)
NameError: global name 'MIN_FILE_SIZE_BYTES' is not defined

shntnu commented 3 years ago

I defined MIN_FILE_SIZE_BYTES, now this

Queue exists
Cluster default exists
Traceback (most recent call last):
  File "run.py", line 597, in <module>
    setup()
  File "run.py", line 347, in setup
    update_ecs_task_definition(ecs, ECS_TASK_NAME, AWS_PROFILE)
  File "run.py", line 138, in update_ecs_task_definition
    task_definition = generate_task_definition(AWS_PROFILE)
  File "run.py", line 124, in generate_task_definition
    "value": str(USE_PLUGINS)
NameError: global name 'USE_PLUGINS' is not defined

bethac07 commented 3 years ago

One often-problematic parameter now auto-calculated
- DOCKER_BASE_SIZE
Five parameters added
- MACHINE_TYPE and MACINE_PRICE - moved in from fleet file
- DOWNLOAD_FILES- to set if you want to download files to EBS rather than load via S3FS (buggy)
- NECESSARY_STRING- when deciding if a job has already been done before, can filter to only certain files
- USE_PLUGINS

shntnu commented 3 years ago

Closing in favor of #59