ERDDAP / erddap

ERDDAP is a scientific data server that gives users a simple, consistent way to download subsets of gridded and tabular scientific datasets in common file formats and make graphs and maps. ERDDAP is a Free and Open Source (Apache and Apache-like) Java Servlet from NOAA NMFS SWFSC Environmental Research Division (ERD).
Creative Commons Zero v1.0 Universal
76 stars 54 forks source link

Watch dir threads #155

Closed ChrisJohnNOAA closed 1 month ago

ChrisJohnNOAA commented 1 month ago

Description

Adds a new approach to watch directories for changes. This utilizes one thread per file system instead of one thread per dataset being watched. This will drastically reduce the number of threads needed for watching datasets. The new approach does mean file change events are handled for all datasets, so updates will happen more frequently (matching the most updated dataset under the old approach, or even faster).

Fixes # (issue)

Type of change

Please delete options that are not relevant.

Checklist before requesting a review

ChrisJohnNOAA commented 1 month ago

Documentation updates are currently just in the 2.24 branch of the github io documentation https://github.com/ERDDAP/erddap.github.io/commit/6482ddd09d31e8e5f9ecb59d2fd5f599bc8ccde7