curate stuff in `simplification_protocol.ipynb`?

[jGaboardi]

• Should the stuff in the simplification_protocol.ipynb notebook be migrated into core/*?
• If yes:
  • Where do we think it should go // be named?
  • Where is the data?

[martinfleis]

Yes.

Where do we think it should go // be named?

viz.protocol?

Where is the data?

Await a PR :)

[jGaboardi]

@Kryndlea

Within the notebook, are the "input_file" the original OSM roads data?

For example:

{'case': 'Case 11 - Cloverleaf interchange', 'input_file':'1133_output.gpkg', ...

Does '1133_output.gpkg' refer to the data now found in: data/1133/roads_osm.parquet?

[jGaboardi]

Plan

• core/protocol.py with:
  • the protocol definitions
  • a top-level prep func (combined version of preparation portions of:
    • create_pictures_original() & create_pictures_simplified() (they are nearly identical)
    • probably rename process_protocol()
  • core/viz/protocol.py
  • plotting portions of create_pictures_original() & create_pictures_simplified() (they are identical)
  • protocol_images/*
  • a top-level directory structured like data/* for storing images
  • got any better ideas?

@Kryndlea @dancejod If yall have any input on this I am happy to hear it.

[martinfleis]

@jGaboardi each of the GPKG contains two layers. Input and output. I thought you were aware since you were converting them to parquet? If not, can you verify that you converted a correct layer?

[jGaboardi]

@jGaboardi each of the GPKG contains two layers. Input and output. I thought you were aware since you were converting them to parquet? If not, can you verify that you converted a correct layer?

Nope, I completely misunderstood. I had thought the .gpkg files were simply a different format of the singular output and saved them back as manual/<FUA>.parquet files. Good thing we caught that now. I'll open another ticket and update the data.

What do we think would be better:

1. two 2 subdirs: manual/input/<FUA>.parquet & manual/output/<FUA>.parquet; or
2. one subdir: manual/<FUA>_input.parquet ... manual/<FUA>_output.parquet

cc @anastassiavybornova

[martinfleis]

We have input already here as roads_osm.parquet, so I think we can ignore the input layer of the GPKG.

@Kryndlea @dancejod the input is not changed in any way, right?

[jGaboardi]

We have input already here as roads_osm.parquet, so I think we can ignore the input layer of the GPKG.

@Kryndlea @dancejod the input is not changed in any way, right?

Ah, yeah. That's what I said above.

So if that's the case, we're all good.

[anastassiavybornova]

hi! just to say that i was aware of this, so the two manually simplified parquet files i uploaded were already preprocessed (extracting the output layer from the gpkg that i got from gdrive). but might be irrelevant by now if we (aka james) will redo all the manual data uploading? anyways. +1 for not adding another subdirectory, and just saving the output layer directly into the manual folder.

[martinfleis]

So if that's the case, we're all good.

the only thing I am not sure is whether you have always read the output layer from the GPKG

[jGaboardi]

I am going to double check what's in there from #71 just to make sure

[jGaboardi]

So if that's the case, we're all good.

the only thing I am not sure is whether you have always read the output layer from the GPKG

Exactly. I thought there was only 1 layer in those, so I wasn't specifying a layer

[dancejod]

We have input already here as roads_osm.parquet, so I think we can ignore the input layer of the GPKG.

@Kryndlea @dancejod the input is not changed in any way, right?

Yes, the input is not changed.

[martinfleis]

I thought there was only 1 layer in those, so I wasn't specifying a layer

GeoPandas 1.0 released earlier today would warn you about that 😉