search

weecology / NEON_crown_maps

Generating tree crown maps for NEON sites
MIT License
1 stars 0 forks source link

scaling dask cluster 403 forbidden #19

Closed bw4sz closed 4 years ago

bw4sz commented 4 years ago

at at 347 it stops. /orange/ewhite/b.weinstein/NEON/crops/2018_OSBS_4_406000_3292000_image.tfrecord

is it the same record everytime?

bw4sz commented 4 years ago

/orange/ewhite/b.weinstein/NEON/predictions/2019_TEAK_4_321000_4092000_image.shp

bw4sz commented 4 years ago

i think @maroniS has some old files on that folder that only he has permissions for. I'm dropping osbs for now.

bw4sz commented 4 years ago

i'm wrong, its not just OSBS, seeing it on BART too. No idea.

bw4sz commented 4 years ago

Inching closer. Cannot reproduce on BART and TEAK at smaller scales (n=5, n=20, n=100). Trying now with entire site.

On Sun, Mar 8, 2020 at 9:38 PM Ben Weinstein notifications@github.com wrote:

i'm wrong, its not just OSBS, seeing it on BART too. No idea.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/weecology/NEON_crown_maps/issues/19?email_source=notifications&email_token=AAJHBLCPZXT3M5Y2C777CJTRGR6GTA5CNFSM4LDV33VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOFT3LI#issuecomment-596327853, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJHBLGL2TUU5MIVIGD4MLDRGR6GTANCNFSM4LDV33VA .

-- Ben Weinstein, Ph.D. Postdoctoral Fellow University of Florida http://benweinstein.weebly.com/

bw4sz commented 4 years ago

It was the urllib checking the deepforest latest release, too many calls to github at scale. Solved by pointing to local model weights and skipping 

model.use_release()