handleSGfindtags(): fetch motusProjectID from appropriate receiver deployment, if any

[jbrzusto]

Ownership of a batch's products follows these rules:

• if the boot session overlaps a receiver deployment record, use that projectID
• otherwise, use the motusProjectID parameter specified in the top-level job, if any

[jbrzusto]

also apply this fix to handleLtFindtags()

[denislepage]

Can you confirm that you are storing the project provided by users during upload?

This is the value that we need to preserve and that should be made available through the API.

Thanks

[jbrzusto]

Assigning the upload-user-provided projectID to a batch doesn't seem to always be possible or correct, if the purpose is to determine ownership of the batch. Here are examples of cases where it doesn't seem to work:

• user uploads files from several receivers bundled into a single archive; this works fine, but what if the receivers don't belong to the same project, as sometimes happen when projects share field duties

• user uploads files from a single receiver for a time period covered by an existing receiver deployment record, but chooses a projectID different from the one for the receiver deployment

• administrator reruns a boot session from a receiver. This boot session will often include files from different uploads, and users might not have assigned the same projectIDs to those, even though ownership of a boot session of data is always by a single project (based on the rule that a receiver must be rebooted between deployments across different projects.)

We can always fall back to the user-supplied projectID for a batch if there is no overlapping receiver deployment already registered with motus. But if the latter exists and the two disagree?

[denislepage]

We are accounting for assigning batches (and each hit) to the most appropriate deployment, but we also need to preserve the project that was supplied by the user. If the 2 concepts get mixed in when we get the data from sgdata, this limits our opportunities to resolve problems.

Motus is the only client pulling data from sgdata, so there is limited need to be concerned about linking batches to projects on the basis of deployments on the SG data server for permission purposes.

This information isn’t static anyway, and we have the processes in place to reassign the batches to the appropriate projects whenever metadata changes.

So, what we need is to have the identity of the projectID that was supplied by the user at the time of the upload, separately from attempts to resolve batches on the basis of deployments. I seem to recall 2 projectID fields in your tables. Is that the purpose? If you want to provide that as a second projectID field, that’s fine.

[jbrzusto]

So what projectID do you want to assign to a batch in these situations:

1. a batch is generated from an upload where the user assigned the upload to project P1 but the batch also processes files from a separate upload that was assigned to project P2?
2. a batch is the result of an administrator re-running a boot session, which might previously have been processed as multiple batches with different project IDs - e.g. due to the process in (1)?

[denislepage]

In all cases where a user-assigned project isn’t defined, you can use projectID = 0 for that variable. The administrator account that I use to pull in data will be able to see it, but we will need to confirm.

I’m not sure I understand conditions under which the same batch can be using files from different uploads (point #1 below)?

From: John Brzustowski [mailto:notifications@github.com] Sent: 13 Mar 2018 08:26 To: jbrzusto/motusServer Cc: Denis Lepage; Comment Subject: Re: [jbrzusto/motusServer] handleSGfindtags(): fetch motusProjectID from appropriate receiver deployment, if any (#386)

So what projectID do you want to assign to a batch in these situations:

1. a batch is generated from an upload where the user assigned the upload to project P1 but the batch also processes files from a separate upload that was assigned to project P2?
2. a batch is the result of an administrator re-running a boot session, which might previously have been processed as multiple batches with different project IDs - e.g. due to the process in (1)?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jbrzusto/motusServer/issues/386#issuecomment-372647756, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AUIfDrR4cgrjA_OPv2-s-v_FK1EFBMjgks5td7rFgaJpZM4SkMb6.