trailbehind / DeepOSM

Train a deep learning net with OpenStreetMap features and satellite imagery.
MIT License
1.32k stars 182 forks source link

Path and file IO changes to make the scripts work on my machine #73

Closed dbdean closed 7 years ago

dbdean commented 7 years ago

I've had to make a few changes to the file IO paths and whether folders can be created or deleted to get these scripts to work on my machine.

andrewljohnson commented 7 years ago

Thanks, I appreciate it.

I'll run your changes on my box next week and verify, then accept the PR.

dbdean commented 7 years ago

@andrewljohnson, have you had a chance to review and/or verify this pull request?

andrewljohnson commented 7 years ago

I have been working on this today, but I found issues with the Dockerfile I was trying to fix. I was surprised you got this far without changing the Dockerfile.

On Sun, Jan 29, 2017 at 6:45 PM, David Dean notifications@github.com wrote:

@andrewljohnson https://github.com/andrewljohnson, have you had a chance to review and/or verify this pull request?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/trailbehind/DeepOSM/pull/73#issuecomment-275969688, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbzf-dQH-3mLOK-DFSK_0ZA97Wj8qcxks5rXU7TgaJpZM4LosLO .

dbdean commented 7 years ago

I didn't have any problems with the Dockerfile last time I built it, at least on the master branch. I'm removing my cache and trying again now to see if anything breaks for me.

andrewljohnson commented 7 years ago

I had to make this change to let "make dev" work (non-cpu):

https://github.com/trailbehind/DeepOSM/commit/38cb5391d3a3853505fb0ba0bf87743d257fc5db

And now I'm working on "make dev-gpu" where I ran into this issue late in the install:

The command '/bin/sh -c apt-get update && apt-get install -y --no-install-recommends --force-yes cuda-core-$CUDA_PKG_VERSION cuda-misc-headers-$CUDA_PKG_VERSION cuda-command-line-tools-$CUDA_PKG_VERSION cuda-license-$CUDA_PKG_VERSION cuda-nvrtc-dev-$CUDA_PKG_VERSION cuda-cusolver-dev-$CUDA_PKG_VERSION cuda-cublas-dev-$CUDA_PKG_VERSION cuda-cufft-dev-$CUDA_PKG_VERSION cuda-curand-dev-$CUDA_PKG_VERSION cuda-cusparse-dev-$CUDA_PKG_VERSION cuda-npp-dev-$CUDA_PKG_VERSION cuda-cudart-dev-$CUDA_PKG_VERSION cuda-driver-dev-$CUDA_PKG_VERSION && cd /tmp && apt-get download gpu-deployment-kit && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100 Makefile:28: recipe for target 'dev-gpu' failed make: *** [dev-gpu] Error 1

Andrew Johnson Founder gaiagps.com http://gaiagps.com

On Sun, Jan 29, 2017 at 6:59 PM, David Dean notifications@github.com wrote:

I didn't have any problems with the Dockerfile last time I built it. I'm removing my cache and trying again now to see if anything breaks for me.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/trailbehind/DeepOSM/pull/73#issuecomment-275970923, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbzf5M_fj1rtM-L7G3l1xG7O60nyPdvks5rXVITgaJpZM4LosLO .

dbdean commented 7 years ago

Hmm, I'm not working on the feature-analysis branch (or GPU stuff at the moment). I'm in master.

I'll try and build on that branch then, see what I get.

andrewljohnson commented 7 years ago

I don't think the branch is relevant though, since the Dockerfiles haven't changed. This branch was just where I left off, I figured I should merge it and then integrate your change if needed.

I may have solved this now too, waiting for the build to run right now...

Andrew Johnson Founder gaiagps.com http://gaiagps.com

On Sun, Jan 29, 2017 at 7:10 PM, David Dean notifications@github.com wrote:

Hmm, I'm not working on the feature-analysis branch (or GPU stuff at the moment). I'm in master.

I'll try and build on that branch then, see what I get.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/trailbehind/DeepOSM/pull/73#issuecomment-275971814, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbzf9LXdZJAO25QXb1-MgMWNDUrI70pks5rXVSKgaJpZM4LosLO .

dbdean commented 7 years ago

I'll wait and see how you go, then :+1:

dbdean commented 7 years ago

The shutil.rmtree one is because on first run the cache directories aren't there yet, and trying to remove them when they don't already exists raises an exception.

The try/except around makedirs is because makedirs raises an exception if the directory already exists (in this case, the parent of the cache_filename).

dbdean commented 7 years ago

Thanks!

On Mon., 30 Jan. 2017, 14:37 Andrew L. Johnson, notifications@github.com wrote:

Merged #73 https://github.com/trailbehind/DeepOSM/pull/73.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/trailbehind/DeepOSM/pull/73#event-940824442, or mute the thread https://github.com/notifications/unsubscribe-auth/AMIsPkLUzxJU-lTpzsmqU96c-Yz-0QV3ks5rXWkBgaJpZM4LosLO .

-- http://dbdean.com

andrewljohnson commented 7 years ago

Likewise!

On Sun, Jan 29, 2017 at 8:41 PM, David Dean notifications@github.com wrote:

Thanks!

On Mon., 30 Jan. 2017, 14:37 Andrew L. Johnson, notifications@github.com wrote:

Merged #73 https://github.com/trailbehind/DeepOSM/pull/73.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/trailbehind/DeepOSM/pull/73#event-940824442, or mute the thread https://github.com/notifications/unsubscribe- auth/AMIsPkLUzxJU-lTpzsmqU96c-Yz-0QV3ks5rXWkBgaJpZM4LosLO .

-- http://dbdean.com

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/trailbehind/DeepOSM/pull/73#issuecomment-275978938, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbzf9AjhvFIpItyjb2p9yYkOaVOEOx7ks5rXWnhgaJpZM4LosLO .