Update docker-compose profiles to match developer use case

[tloubrieu-jpl]

We identify n configuration that a developer/tester need to start the registry with:

• Local working registry for a user who want to experiment with a local registry.

• Automated Integration tests: Start the registry with a reference dataset and run reference test suites (from a postman collection)

• Specific test (e.g. Reproduce a user error):

  • Start an empty registry (elasticsearch and api)
  • Load a specific dataset to the registry to reproduce an error of a user

• API developer: start the registry with a elastisearch started with a given reference dataset. The api run in the IDE not with docker-compose. The Developer will update the integration tests (To Be defined)

• Harvest developer: Start the registry with elasticsearch and api service to control that new loader development work. The loader run in the IDE, no with docker compose. The Developer will update the integration tests (To Be defined)

A break down of the profile is proposed on https://docs.google.com/spreadsheets/d/1jTP_21jIW9wLnncdAKZgeIESf5eI4HSHQCAe3mVVG-E/edit#gid=1990932397

[tloubrieu-jpl]

@al-niessner , @tdddblog , @jordanpadams , to finalize the work that @ramesh-maddegoda made with docker-compose on the registry, I would like to identify, the different configuration in which a developer/tester would need to start the registry (which is the loader + elasticsearch + api components).

@al-niessner, I remember you run your IDE in docker, I hope that will not work for you as well.

Can you all comment this ticket and check that it covers your needs ?

Thanks

[jordanpadams]

@tloubrieu-jpl @al-niessner @tdddblog @ramesh-maddegoda here are the set of deployments I see from a node perspective. There may be some developer deployments we are missing, and not sure if I have the new Harvest Client / Server architecture correct.

@ramesh-maddegoda I've just added you to the EN team, so be sure to sign-up with your JPL email address @jpl.nasa.gov). Not sure if I mentioned before, but linking your @jpl.nasa.gov to a new public Google account is probably a good idea moving forward since we do a lot of work out in the open that require a public-facing Google account. See this doc for linking the two.

https://lucid.app/lucidchart/72d50ed1-db7f-48cf-98ee-5259f1d57922/edit?invitationId=inv_388716fc-dbb8-4d45-afb2-40d1135b683f

[tloubrieu-jpl]

@ramesh-maddegoda will work on that after the dockerization of bigdata harvest #200 is done.

The use cases will need to be revised to integrate the new bigdata ingestion tools