Using different (external) dataset with different image resolution

[rahul-nt]

Hi I'm trying to run the code for the road extraction dataset from DeepGlobe challenge, where the images are 1024x1024x3, and (even though from 3 different regions but) have been named independent of the regions. It seems to me that the code however is written in accordance with the data acquired from the google maps. Are there any instructions regarding as to with what changes the code should be run to make it work for any external dataset?

[uakfdotb]

For this code, each tile could be a separate region, so name it like this:

r000_0_0_sat.png r001_0_0_sat.png etc.

(all x=0, y=0)

Then you'll need to reformat the road network graphs to be in the graph file format that RoadTracer expects. You can see examples of the expected file format in https://roadmaps.csail.mit.edu/roadtracer/graphs.zip. Essentially, there is a section of vertices, followed by a blank line, followed by a section of directed edges (but the RoadTracer code expects an undirected graph where directed edges always go in both directions). Each vertex is annotated with a spatial position.

In these files, the vertex position should be a pixel position in the imagery, with (0, 0) at the topleft of the image. So if your tile is 1024x1024 with road from (100, 100) to (100, 500), then the graph file looks like this:

100 100 100 500

0 1 1 0

Then you can name the files like this:

r000.graph r001.graph r002.graph etc.

Next you need to make a couple updates to the tileloader.py. Set tile_size to the side length of your square tiles, and update TRAINING_REGIONS to be a list like this:

['r000', 'r001', 'r002', etc.]

I think you will need to make additional updates to dataset/lib/regions.go so that you can run 4_startlocs.go, which generates two JSON files that are also needed as input for the roadtracer/train.py script.

[rahul-nt]

@uakfdotb Thank you so much for the reply. I will try this out.

[rahul-nt]

Hi I tried to look at the things more closely after your reply. If I understand correctly, the roadtracer code creates the labels/masks (the ground truth) for the training images by running the go scripts in the dataset directory. But what if we have already have the satellite images as well the mask images [the ground truth] (as it is in the case of DeepGlobe's dataset)? Because, from your reply, what I understood is that the graph files as well as the json files are important for running the training and eventually the prediction.

So, is there away a reverse also possible to generate the graph files from the already available ground truth mask files?

[uakfdotb]

The roadcnn code (not RoadTracer system, but rather a baseline from the paper) uses the masks produced by dataset/5_truth_tiles.go, but RoadTracer doesn't actually use the masks at all, only the graphs.

So if you only have masks, you could use them directly with roadcnn, but not with roadtracer.

utils/mapextract.py will take an image and produce a graph from it via thresholding (binary mask from 8-bit image given a threshold), morphological thinning (thin foreground regions until everything is single-pixel-wide lines), and extracting a graph by following the remaining foreground pixels.

This utility is included to convert outputs from roadcnn to vector graph file format. But you could use it to convert the DeepGlobe masks to ground truth graphs as well.

One thing to keep in mind is that RoadTracer never segments the imagery, it produces a graph directly. So you won't be able to compare it on DeepGlobe's intersection-over-union metric. This is a bad metric anyway because it only looks at the segmentation output, not the actual connectivity / topology of the graph (e.g., even if you have many disconnections your performance on metric might still be good, but the map won't be usable for any practical application).

[anilbatra2185]

Hi @uakfdotb, Thanks for sharing the code, a unique way to generate the graph. I tried to follow the same steps for DeepGlobe dataset and trying to modify the code in files. I summarized the modifications steps as follows:

region.go

• GetRegions method from lib will always return only region = 'r0001', radiusX=1, radiusY=1, centerGPS=None, CenterWorld=None.

4_startlocs.go

• Remove the loop at line 99 for radiusX/radiusY and simple process the tiles with rectangle as (0,0)-(1024,1024).
• Also, remove the subtraction of 256 from rectangle coordinates in all files.
• Also, I think there is no need to scale with 4096.

Can you please verify this and let me know if there is any more changes required in any other file?

Thanks

[uakfdotb]

anil-2185 Those two modifications sound good. You would directly create the ground truth road network graphs in pixel coordinates based on DeepGlobe dataset, right?

You will have to make some similar adjustments to roadtracer/tileloader.py. Inference requires one point known on the road network, for now maybe split the training set from DeepGlobe into train and test set. But later you could apply on DeepGlobe test set by using peaks in the segmentation output (from another model, e.g. roadcnn) as starting locations for RoadTracer.

[rahul-nt]

Is it possible to run roadtracer inference on DeepGlobe data, using the pre-trained model and graphs (and the generated json) from this link: https://roadmaps.csail.mit.edu/roadtracer/

I tried looking at infer.py and tileloader.py, and I'm not sure where to start from to makes changes to accomplish the above. Also, is the documentation for the code available?

[anilbatra2185]

anil-2185 Those two modifications sound good. You would directly create the ground truth road network graphs in pixel coordinates based on DeepGlobe dataset, right?

You will have to make some similar adjustments to roadtracer/tileloader.py. Inference requires one point known on the road network, for now maybe split the training set from DeepGlobe into train and test set. But later you could apply on DeepGlobe test set by using peaks in the segmentation output (from another model, e.g. roadcnn) as starting locations for RoadTracer.

Thanks @uakfdotb for response. I am working on the modifications to train the roadtracer on DeepGlobe. As suggested I am able to run the code on Deepglobe, but has few doubts:

• I am not able to understand the subtraction of number 256 in "4_startlocs.go". But found that subtraction is considered necessary in tileloader.py and train.py. Is it required to set the path (start and end rectangle) in the center of image?

• Window size (search area, I assume) is 512 in tileloader.py but in train.py is 256 and model also expects input of size (?,256,256,5). should the window size in tileloader.py be also 256?

• As in DeepGlobe, number of testing tiles can be more than 1, as compare to single testing tile of Chicago. Can you please suggest the changes for the same in train.py? I tried changing the code similar to train_tiles and create list of test_tiles but I am struggling to run the testing stage...any help is appreciated.

Here are few input images to model while training for the verification.

Thanks

[anilbatra2185]

Hi @uakfdotb I am able to execute the code on DeepGlobe dataset and attached few inferred roads on the imagery. I have few more doubts:

• How to visualize the graph over the image, as I am not able to get any clue to change the CreateSVG method in Common package?
• While inferring only single starting location is randomly chosen from the ground truth (consider for validation set). Thus I am unable to infer the road network completely. How did you rectify this while testing? an example image is shown below with ground truth and inferred roads starting with 2 random locations.

Ground Truth (Segmentation Mask) With Starting Location 1 With Starting Location 2

Thanks Anil

[uakfdotb]

I am not able to understand the subtraction of number 256 in "4_startlocs.go". But found that subtraction is considered necessary in tileloader.py and train.py. Is it required to set the path (start and end rectangle) in the center of image?

That code is to make sure there are enough starting locations in the tile to use it for training. Starting locations near the border of the image will be excluded because during training we take crops of the imagery and there is no code in place to allow crops that go off of the image canvas. You could modify the training/inference code to allow these crops and set the pixels to 0 if they are outside the image bounds.

Window size (search area, I assume) is 512 in tileloader.py but in train.py is 256 and model also expects input of size (?,256,256,5). should the window size in tileloader.py be also 256?

It should be okay to set it to 256, it just needs to be larger than the actual window size.

As in DeepGlobe, number of testing tiles can be more than 1, as compare to single testing tile of Chicago. Can you please suggest the changes for the same in train.py? I tried changing the code similar to train_tiles and create list of test_tiles but I am struggling to run the testing stage...any help is appreciated.

For DeepGlobe you should make each image a separate region, and it can be BLAH_0_0.png. (because the images are not contiguous, right?)

How to visualize the graph over the image, as I am not able to get any clue to change the CreateSVG method in Common package?

It may be easier to write your own code. You can see discoverlib/graph.py to load the graph, and then the vertex.point Point objects just reference pixel positions in the imagery. So you can draw lines along edges using your desired programming language / library.

While inferring only single starting location is randomly chosen from the ground truth (consider for validation set). Thus I am unable to infer the road network completely. How did you rectify this while testing? an example image is shown below with ground truth and inferred roads starting with 2 random locations.

The most straightforward way would be to first run a segmentation model, and then take maximum value in the segmentation output as starting location.

[uakfdotb]

More on using segmentation approach:

After running segmentation model, use the maximum value pixel in the segmentation output as starting location for RoadTracer. Then, run RoadTracer to termination. Next, zero out all pixels in the segmentation output that are close to any edge in the output of RoadTracer so far. If the maximum value pixel is above a certain threshold (e.g. 0.5), then repeat the process.

This way, if there are two sets of disconnected roads in the image, we can run RoadTracer on both sides of the road network.

Alternative approach: run one iteration of RoadTracer from every grid vertex (20, 20), (20, 40), (20, 40), (40, 20), (40, 40), etc. If the angle output from RoadTracer is above a high threshold (e.g. 0.5 or 0.6) then run RoadTracer to termination with a normal threshold (e.g. 0.3 or 0.4). Then repeat only on the grid vertices that are not within 100 or so pixels of the road network inferred by RoadTracer so far.