Rec Model 2.0

[davemfish]

We have some funds to do some rec model development; see https://github.com/natcap/softwareteam/issues/143

The objective is to integrate an archive of geotagged tweets to supplement the flickr photo dataset we use now.

There will be some updates to the server module to accommodate source data in a series of CSV files, instead of one big CSV.

And there will be client updates to the linear regression model, to use twitter-user-days as well as photo-user-days as the response variable.

[davemfish]

• [ ] Update tests
• mostly done. regression data will need updating after linear regression changes are made
• [ ] Check docstrings
• mostly done. do another pass before PR.
• [x] Update MODEL_SEPC
• also depends on linear regression changes, for descriptions of output files
• [x] Update linear regression specification based on design doc feedback
• (awaiting more feedback)
• [x] Restrict extent of the AOI to avoid unreasonably large queries. Use the quadtree to count number of points in all nodes intersecting the AOI bounding box. This will be an upper bound for the number of points within the AOI.
• [x] results_suffix is bugged
• [ ] re-expose the workspace_fetcher through the RecManager , or deprecate it.
• [ ] switch from shapefiles to geopackage; finalize output fieldnames
• [ ] update sampledata JSON with new min_year, max_year values

Server infrastructure:

• [ ] Decide on appropriate specs (mainly number of cores)
• [ ] Consider running the new server (flickr + twitter) on the same VM as the old server, which we'll need to keep running for compatibility with older invest versions
• [ ] cleanup the server registry listings in data.naturalcapitalproject.org/server_registry (for the new model, consider moving to natcap-recreation bucket).

[davemfish]

Is there a race condition in quadtree construction?

08/22/2024 20:13:38  natcap.invest.recreation.recmodel_server INFO     zipping result
08/22/2024 20:13:39  natcap.invest.recreation.recmodel_server ERROR    calculate_userdays exited while multiprocessing.
[Errno 17] File exists: '/home/davemfish/server/flickr/local/a9c0e739-85f0-494d-aa8c-da1bd2edf020/local_qt/3a'
concurrent.futures.process._RemoteTraceback: 
"""
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/davemfish/invest/env/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/davemfish/invest/env/lib/python3.11/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
           ^^^^^^^^^^^^^^^^
  File "/home/davemfish/invest/env/lib/python3.11/site-packages/natcap/invest/recreation/buffered_numpy_disk_map.py", line 129, in _write
    os.mkdir(array_directory)
FileExistsError: [Errno 17] File exists: '/home/davemfish/server/flickr/local/a9c0e739-85f0-494d-aa8c-da1bd2edf020/local_qt/3a'
"""