bcgov / FIT_changedetector

Compare two sets of geodata, reporting on various types of differences
Apache License 2.0
2 stars 0 forks source link

compare command - outputs #8

Open smnorris opened 1 month ago

smnorris commented 1 month ago

Currently, the compare command always outputs to changedetector.gdb, defaulting to cwd but with --out-path option

andershopperstead commented 1 month ago

Specifying output folder is probably sufficient for now, but could you please include a default timestamp to the end of the changedetector name in _YYYYMMDD_HHMM format. We have started using this protocol with other tools and it is helpful to track work and prevent overwrites. Thank you.

With regards to other output formats, GDB is good for now but our team is going to discuss what we might like to support in the future. Thanks for your patience.

smnorris commented 1 month ago

Sounds good, thanks.

Note that writing non-spatial tables to .gdb doesn't seem to be supported by the default writer (pyogrio). It might just be a configuration option though - and if not, it shouldn't be too hard to work around to get this working. But I'll wait on feedback on what file types might be supported - maybe non-spatial comparisons do not have to be written to .gdb at all.

smnorris commented 1 month ago

Default output is now changedetector_YYYYMMDD_HHMM.gdb in current working directory. I've changed the out_path option to out_file to handle this, not all tools will want the timestamp (eg - the open data downloader writes to object storage, where versioning is handled automatically)

andershopperstead commented 4 weeks ago

Re: Other formats, confirmed the GDB is sufficient for now. We can keep these options in the backlog as they may be needed in the future.

Re: change of the out_path to out_file, I am unclear on the rationale for the change you made. The timestamp was requested for file management purposes, not for versioning reasons. Can the user still enter a directory for the out_file parameter?

smnorris commented 4 weeks ago

Re: Other formats, confirmed the GDB is sufficient for now. We can keep these options in the backlog as they may be needed in the future. 👍

Re: change of the out_path to out_file, I am unclear on the rationale for the change you made. The timestamp was requested for file management purposes, not for versioning reasons. Can the user still enter a directory for the out_file parameter?

If the tool defaults to a filename that is not consistent (ie with a timestamp), there needs to be an option for specifying some other output filename - for cases where a timestamp is not preferred.

The --out_file option is one way to get there - it accepts a full path - C:\path\to\my\output.gdb or C:\path\to\my\output_20241018_1155.gdb etc - but to get the timestamp, the user has to add it themselves.

To get the tool to write to a default filename with timestamp in a user specified folder, we could do one of:

  1. create the out_file name with timestamp in the arcgis wrapper (or whatever else is calling the the script)
  2. split the --out_file into two parameters, --out_path and --out_file, and when just --out_path is provided, default to writing changedetector_<timestamp>.gdb in that path
andershopperstead commented 4 weeks ago

Thanks so much for the explanation of the issue and options. We want to avoid having the user add the timestamp as a parameter. Given that, I think option #2 in your most recent comment would be our preference.