search

jhuapl-boss / boss-manage

Repo for Boss configuration and management software
Apache License 2.0
5 stars 6 forks source link

Downsample queue #92

Closed movestill closed 3 years ago

movestill commented 3 years ago

Add queue for downsample jobs so the user can request a downsample and walk away if another is already running.

Sequence diagram here: https://github.com/jhuapl-boss/boss-manage/blob/downsample-queue/docs/downsample/downsample-sequence-diagram.png

Related PRs:

https://github.com/jhuapl-boss/boss/pull/63 https://github.com/jhuapl-boss/boss-tools/pull/40

movestill commented 3 years ago
Screen Shot 2020-11-18 at 5 39 38 PM

This is a screenshot of an execution of the updated downsample step function.

dxenes1 commented 3 years ago

Another general question I have, can we start downsamples with base resolutions other than 0 now? I remember this being an issue before but I don't know if it was fixed. For 99% of the cases it doesnt matter but when uploading like a mip3 dataset on cloudvolume it would be nice to conserve that as res3 on bossdb.

movestill commented 3 years ago

Another general question I have, can we start downsamples with base resolutions other than 0 now? I remember this being an issue before but I don't know if it was fixed. For 99% of the cases it doesnt matter but when uploading like a mip3 dataset on cloudvolume it would be nice to conserve that as res3 on bossdb.

I didn’t touch any code related to the base resolution, so not sure.

perrywilson commented 3 years ago

I read through all of the code. Not sure I understand the heaviside DSL but the changes seem to make sense. I also learned a bit about step functions and SNS vs SQS services.