schemas: are external_system_numbers obsolete?

[jmartinm]

The field 035__z contains some obsolete ids currently in production - see https://github.com/inspirehep/inspire-next/blob/master/inspirehep/dojson/current_marcxml_usage/hep.txt#L957-L970

When migrating records to the new system, the values are being lost at the moment since we do:

return {
            'value': value.get('a'),
            'institute': value.get('9'),
            'obsolete': bool(value.get('z')),
        }

So some examples of entries are:

"external_system_numbers": [
                  {
                     "obsolete": false,
                     "institute": "INSPIRETeX",
                     "value": "Adrian-Martinez:2015nin"
                  },
                  {
                     "obsolete": true,
                     "institute": "DESY"
                  }
]

In my demo records all the ones coming from DESY appear like that.

So questions:

1) Do we want to keep the obsolete flag but still fill the value? That will imply modifying the logic in our templates to display value if not obsolete

2) Do we like the way obsolete is modeled at the moment? One good thing is that when editing a record, close to the value you will have a checkbox to mark as obsolete (if we keep it as a boolean) so that is quite nice.

Any thoughts? If green light, I can implement 1)

@inspirehep/inspire-content

[annetteholtkamp]

We should definitely keep the value. And I like option 2.

• Annette

On 05 Aug 2016, at 08:59, Javier Martin Montull notifications@github.com<mailto:notifications@github.com> wrote:

The field 035__z contains some obsolete ids currently in production - see https://github.com/inspirehep/inspire-next/blob/master/inspirehep/dojson/current_marcxml_usage/hep.txt#L957-L970

When migrating records to the new system, the values are being lost at the moment since we do:

return { 'value': value.get('a'), 'institute': value.get('9'), 'obsolete': bool(value.get('z')), }

So some examples of entries are:

"external_system_numbers": [ { "obsolete": false, "institute": "INSPIRETeX", "value": "Adrian-Martinez:2015nin" }, { "obsolete": true, "institute": "DESY" } ]

In my demo records all the ones coming from DESY appear like that.

So questions:

1) Do we want to keep the obsolete flag but still fill the value? That will imply modifying the logic in our templates to display value if not obsolete

2) Do we like the way obsolete is modeled at the moment? One good thing is that when editing a record, close to the value you will have a checkbox to mark as obsolete (if we keep it as a boolean) so that is quite nice.

Any thoughts? If green light, I can implement 1)

@inspirehep/inspire-contenthttps://github.com/orgs/inspirehep/teams/inspire-content

— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHubhttps://github.com/inspirehep/inspire-next/issues/1405, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AM1-OwxQku2VLXfcgrczvawXM_CG2Dmgks5qct9sgaJpZM4JdaIY.

[jmartinm]

We should definitely keep the value. And I like option 2.

👍

So the only thing missing is adding the value to the obsolete fields (and looking if any templates affected)

[ksachs]

001386663 035__ $$9DESY$$zDA15-kp33q are book-keeping numbers from DESY-keywording. I.e. they are internal identifiers, I would not call it obsolete.

001425858 035__ $$9INSPIRETeX$$zSkovpenonbehalfoftheCMSCollaboration:2016pqi are obsolete tex-keys, i.e. obsolete external identifiers.

I believe tex-keys ($$9INSPIRETeX / $$9SPIRESTeX) are the only cases where $$a means 'current value' and $$z means 'obsolete'. For all other cases I don't see a difference between $$a and $$z.

[kaplun]

So it's more:

• if $z
  • if $9INSPIRETeX:
  • obsolete = True

in every other case we copy either $z or $a and simply at display time, we remember that DESY's IDs should not displayed. OK? IMHO it's not worth introducing a new flag internal since this can be derived from the $9 value.

[annetteholtkamp]

We also have Fermilab rep nrs in 037__z. They are not obsolete but serve only for internal Fermilab bookkeeping. They shouldn’t be displayed.

• Annette

On 09 Aug 2016, at 08:31, Samuele Kaplun notifications@github.com wrote:

So it's more:

if $z if $9INSPIRETeX: obsolete = True in every other case we copy either $z or $a and simply at display time, we remember that DESY's IDs should not displayed. OK? IMHO it's not worth introducing a new flag internal since this can be derived from the $9 value.

— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub https://github.com/inspirehep/inspire-next/issues/1405#issuecomment-238488128, or mute the thread https://github.com/notifications/unsubscribe-auth/AM1-O0vVQ7s9TGgztK3IfnYs1B7sZBOhks5qeDrbgaJpZM4JdaIY.

[kaplun]

Are they tagged as Fermilab somehow?

[ksachs]

what about flagging everything in 037__z hidden or internal or ... instead of 'obsolete'. Meaning they are indexed but not shown in formats.

[hoc3426]

All Fermilab report numbers in 037__z start with "FERMILAB-" and they should be included int he report number index but not displayed.

[jacquerie]

I think that the answer to the OP is "no", but @michamos should probably take a look and maybe a decision here.

[michamos]

The current situation is as follows:

• for texkeys, we have a texkeys field: the first element comes from $$a, the next from $$z;
• for arXiv identifiers, we have an arxiv_eprints field unifying 035, 037 and 65017;
• for the other external system numbers, we have an external_system_identifiers field, depending on the value of $$9, we populate the value either from $$a or $$z (see https://github.com/inspirehep/inspire-dojson/blob/master/inspire_dojson/hep/rules/bd0xx.py#L160-L210);
• for report numbers, we have a report_numbers field: there is a source and a hidden flag besides the value, that allows to distinguish between $$a and $$z.

This should capture all usecases. If you think something is missing, please comment and/or reopen the issue.