BYO nginx

[florianm]

Hi, sorry to pile this onto a young project... Absolutely great to see a security-first focus and automation in this package!

However, I'm trying to integrate ODK Central into an existing (Docker-based) infrastructure with an existing nginx reverse proxy, which handles SSL for us. The built-in nginx and its SSL handling did not work at all with our own layer of nginx goodness.

Is there a simple way to deploy ODK Central through docker-compose without the baked in nginx and its SSL certificates? Would it be as simple as modifying docker-compose.yml, dropping the nginx service, and exposing 8383 on the service service?

[issa-tseng]

You won't want to drop the nginx service; it is both an SSL termination point as well as a reverse proxy mux to the frontend/backend.

I think if you look in files/nginx and check the two files there it will become fairly clear what to do:

• The sh script can probably reduce to nginx -g "daemon off;"
• The nginx config can listen on port 80 instead of 443 etc.
• I think you'll also have to go to nginx.dockerfile and add a line to manually delete /etc/nginx/conf.d/certbot.conf because that file tries to listen on 80 and redirect http to https.
  • Or you can do this from within the shell script.

[florianm]

Thanks @issa - so this goes deeper than I thought. Let me meditate on your instructions a bit. I think a common use case for larger organisations will be to supply their own SSL/HTTPS forwarding (to an internal HTTP port of ODK Central, keeping the frontend/backend mux), database and mailserver. If those settings could be split out as env vars to the compose file, this would be amazing. If I get something to work along those lines I'll send a PR.

[issa-tseng]

the mail server should already be customizable, at least. the service configuration will allow you to point at any valid smtp relay on any reachable host.

likewise for the database server, though i have not tried this. On Thu, Aug 30, 2018 at 22:29 Florian Mayer notifications@github.com wrote:

Thanks @clint-tseng https://github.com/clint-tseng - so this goes deeper than I thought. Let me meditate on your instructions a bit. I think a common use case for larger organisations will be to supply their own SSL/HTTPS forwarding (to an internal HTTP port of ODK Central, keeping the frontend/backend mux), database and mailserver. If those settings could be split out as env vars to the compose file, this would be amazing. If I get something to work along those lines I'll send a PR.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/opendatakit/central/issues/17#issuecomment-417556048, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHMAqCqkF6OC4grW4ej9GoYq8ivhZ0nks5uWMnIgaJpZM4WTBOC .

[issa-tseng]

hey florian: did you ever get anywhere with this? in light of the discussion on #22 i want to see how others are doing with off-the-beaten-path customizations and how we might maintain some sort of "advanced users/at your own risk" alternative deployment setup.

[florianm]

Hi @issa - sorry, have been sidelined to other projects. I intend to evaluate the ODK 2 suite early 2019, won't be able to work on this earlier unfortunately... our infrastructure is in flux atm with our IT mob deciding between local bare metal and diverse cloud services. Seeing that our internal infrastructure headache is both unpredictable and conflicting with ODK Central's vision, my direction for evaluating ODK Central will likely be a self-hosted solution more in line with the original setup.

[issa-tseng]

hey, no problem at all. just checking in. i would be curious, however, what your goals are with odk2, if you'd be willing to chat about that sometime.

[florianm]

Absolutely! I'll be on the call on 5 Dec.

My long term plan with ODK 2 / general wish list is to close the last few gaps we have with ODK 1:

• Re-visit existing points and then capture new records (ODK forms) about those things with a pre-existing ID. Examples: inventory like park benches in national parks, or marked turtle nests.
• Have a custom form screen workflow (dive into any of say 6 sub-groups in random order), main gripe is the "Add another XXX group" UX for repeating screens.
• Deploy to hundreds of devices in the hands of not always tech-savvy operators and maintain forms (updates) across that fleet of devices. Not sure whether ODK2 fills that gap or ODK 1 with "auto-check/update" and anonymous form submission to ODK Aggregate would do that too.
• ODK Scan looks like a dead sexy way for entering hand-written data sheets.
• QA data captured between say a dozen devices in the field while still offline. Looking at you, ODK sync-endpoint and ODK Tables! Main use case: proof-read typos on hand-written data sheets while memory is still fresh and original data collectors are still around.

As for ODK Central, the most sensible way forward for us would indeed be to follow the plain vanilla deployment instructions (DigitalOcean) and evaluate ODK Central from there.

[issa-tseng]

so, your third bullet we should be addressing soon in Central/Collect, hopefully. we should be talking a little bit about that on the talk tomorrow.

bullets 1 and 2 i am interested in in general, though you're familiar with the areas i touch and Collect isn't really one of them, so i can't make any personal or odk-backed promises. but, again: interested. how do you imagine the identification process happens for a revisit, and how does that relate to the presentation of a form? also, how do you imagine the data comes out the other end such that you can recover that correlation? and for the 6 subgroups, do you basically just mean the form can be sectionally navigated and when you complete a section you return to the section selection, basically? how might the repeating screen ui work better for you?

odk scan i don't know much about, and the odk tables case is definitely the full-blown exactly-what-odk2-was-designed-for case.

[florianm]

Looking forward to that (it's already tomorrow here down under)!

Some ideas on re-visiting: Our static assets we'd like to navigate back to are e.g. a picnic table with a QR code on a metal plate, or a turtle nest with a similarly labelled star picket stuck in the sand (about 1m away inland, so we don't pierce the eggs). The nest tag is a hand-written tag like "2018-12-01 S2N3".

To nagivate back, we would like the app to lead us back to within about 10m of the asset - that's close enough to be in visual range. The label or tag on the asset then serves to identify it (don't need geolocation for that). The navigation back requires offline maps (like mapit GIS) but displaying previously collected ODK records (cross-device sync, worst case while offfline in the field).

The one design decision I'm proud of is that we never update existing forms when re-visiting a previously recorded asset. Instead, we always record new forms when encountering something. The database downstream then links encounters (between unrelated ODK forms) with 'the same thing' by the recorded label (which should, typos aside, exactly match). Therefore, we don't need ODK to identify assets by location, and only to navigate us back to within visual range. I'm not sure whether this needs to happen within ODK Collect (main menu: sync forms, map/list/search/navigate to forms) or a separate app along the lines of "map/list/filter/navigate" like https://data.marinemammals.gov.au/nmmdb

Re repeating groups, I remember @lognaturel leading a UX/UI brainstorming session a while back, coming up with some nice mock-ups. I think the ODK 2 sub-forms are exactly what I'm after - one master screen with six buttons "add another subform X", and a way to list/edit the already captured sub-forms from the master screen. If we were to modify ODK Collect to support these groups, there could be a widget "inline group" that renders:

• a button "add another X" (with custom label)
• a list of buttons for each captured group (tap to edit)

This shouldn't require a change to the data model, right?

[issa-tseng]

thanks for the notes! i'll digest.

as for your final question: i don't pretend to know nearly enough about javarosa to guess. :)

[florianm]

Here's a quick mockup:

[lognaturel]

I’m currently on maternity leave but @smoyth and @cooperka are starting work in the direction of amending how repeats can be added.

The first step is cleaning up the hierarchy view. You can find the spec at https://github.com/opendatakit/roadmap/issues/19 and an open pull request at https://github.com/opendatakit/collect/pull/2740

The adding and removing repeats would be a next step with some discussion both on that roadmap issue and on the forum. The forum would probably be the best place to nail down the intended behavior so it can then be written up as a spec.

CC @yanokwa

[smoyte]

We have PR's ready that will add toolbar buttons for adding and removing repeats. In our design they appear in the hierarchy view. Stay tuned. Still waiting on a code review of the first PR so that we can open the following one.

[florianm]

Hi all, FYI my email woes just rectified themselves with a fresh build of the Docker images, deployed to a different k8s namespace.

This setup works on my end using Azure/k8s/Rancher.

I'm closing this issue, as my original questions/problems are all addressed, but would love to pick up the above conversation about UX with you all.