search

connectome-neuprint / neuPrintHTTP

Implements connectomics REST interface
5 stars 2 forks source link

configuration mechanism changes #53

Closed stuarteberg closed 2 years ago

stuarteberg commented 2 years ago

We need to be able to edit the neuroglancer and skeleton viewer configuration without rebuilding the database.

@umayaml @neomorphic

neomorphic commented 2 years ago

@stuarteberg, This will involve placing that configuration in a file with the neuprintHTTP server, because the neo4j database has to be read only in production or it can potentially be deleted by outside users. We will then need to create an endpoint that can read the files based on dataset, since a neuprintHTTP server can host many datasets. This will also mean that the neuroglancer configuration is not necessarily hosted in the same location as the data. The other alternative is to pay for the enterprise version of neo4j which would allow us to have read only accounts. My guess is that is not an option.

stuarteberg commented 2 years ago

We will then need to create an endpoint that can read the files based on dataset

Can the files just be read once, upon startup?

The other alternative is to pay for the enterprise version of neo4j

Yeah, it looks like our use-case is too big to be hosted on a lower-tier "AuraDB" instance, but the higher-tiers and enterprise editions are too expensive to justify for something like this.

neomorphic commented 2 years ago

We will then need to create an endpoint that can read the files based on dataset

Can the files just be read once, upon startup?

Yes, provided we are happy rebooting the server when we make a change to the layers.

I presume this means we are happy with the burden of managing data for a dataset in different locations.

stuarteberg commented 2 years ago

I presume this means we are happy with the burden of managing data for a dataset in different locations.

Don't worry, it's not "data" -- it's metadata. :-)

But seriously, isn't this just akin to other kinds of server configuration info? For instance, DVID needs a TOML file to boot up. That configuration info doesn't live in leveldb or whatever.

When it comes to "burden", we have to pick our poison. Rebuilding the dataset for every little change we want to make would be a burden. Adding multiple sources of configuration would be a burden. Paying for neo4j enterprise edition would be a burden. Isn't software development mostly about picking the least-worst solution?

neomorphic commented 2 years ago

Completed in 286c9433a2c3e9d97fbf79b77f98594fb781e856