ianwallen commented 3 years ago

When importing data from FGP, we see that the resource descriptions seems to contains multiple fields.

The same field on FGP looks like the following

I don't believe that using structured data into a free text description fields is part of the ISO standard however it is what it is! If not using the same form as FGP, it makes it near impossible for people to know that formatted text needs to go into the description.

Any ideas on how this will be supported in this profile?

[x] 1 - The selected values is using the text instead of the code
[ ] 2 - languages should be a multi-selection option
[x] 3 - The items should be semi-colon separated without space, values should be comma seperated.
[x] 4 - The formatting seems to be missing the indentations. It should be lined up with the other options.

jodygarnett commented 3 years ago

The same issue, placing multiple "fields" into a free form text, was used for organization name:

UI: A custom control was made to support this, it may be used here also?
Validation: I think schematron rules are used to check that this structure, separated by ; is maintained.

jodygarnett commented 3 years ago

I think the same approach can be followed, updating the ticket to describe the work needed.

jodygarnett commented 3 years ago

Aside: I find it odd to see the structured field capturing pdf format, as format is already available is it not?

         <gmd:distributionFormat>
            <gmd:MD_Format>
               <gmd:name>
                  <gco:CharacterString>ZIP</gco:CharacterString>
               </gmd:name>
               <gmd:version>
                  <gco:CharacterString>6.0.4</gco:CharacterString>
               </gmd:version>
            </gmd:MD_Format>
         </gmd:distributionFormat>

But as you say if this is what FGP is checking we will need a hand doing the data entry.

ianwallen commented 3 years ago

gmd:MD_Format is for the gmd:distributionFormat which is separate from the gmd:CI_OnlineResource so I'm not sure what a full sample would look like?

Also, it seem that even if we did identify a better location to put the data, HNAP has already made the decision to put it in this format. The only thing we can hope is that they have changed this hack in future releases of HNAP.

jodygarnett commented 3 years ago

Some discussion:

Looking at using combiner based on organization name example:

{
  "combiner":"; ",
  "root_id":"14",
  "refs":
         {
           "eng":"15" # trying it out with one language
         }
  "defaultLang":"eng",
  "values":
      {
        "eng":"Supporting Document;PDF;eng,fra"
      },
     *
  "config":
  [
      {
        "type": "thesaurus",
        "heading": {
          "eng": "Supporting Document"
        },
        "thesaurus": "external.theme.EC_Resource_Content_Types"
      },
      {
        "type": "thesaurus",
        "heading": {
          "eng": "PDF"
        },
        "thesaurus": "external.theme.EC_Resource_Format"
      },
      {
        "type": "thesaurus",
        "heading": {
          "eng": "eng"
        },
        "thesaurus": "external.theme.EC_Resource_Language"
      },
  ]
}

Approach:

Update resources to use multi-combiner as an experiment that does not require change to core-geonetwork
Use current multi-combiner, but restrict to 1 language while we experiment
If it is required to use two languages (eng, fra, eng,fra) we will need to improve multi-combiner component in core-geonetwork

ianwallen commented 3 years ago

@jodygarnett I think that is a good idea.

I don't understand the restriction to 1 language? It should work with multiple languages the same way the org selector does.

jodygarnett commented 3 years ago

I don't understand the restriction to 1 language? It should work with multiple languages the same way the org selector does.

This was just intended to limit scope to test if the multi-combiner control would work.

jodygarnett commented 3 years ago

I started an experiment with multi-combiner here: https://github.com/metadata101/iso19139.ca.HNAP/tree/description_combiner

Approach:

My approach is:

add settings to enable this approach
first of all get the combiner to work, splitting the free text into three freetext fields (it makes a json map to send to the client) and joining them back up again
Start to update to use thesaurus control

Feedback:

Review with Jose indicates that the multi-combiner may not be usable out-of-the box, if I understand correctly this is due to the description control being on a panel.
Confirmed that the UI approach for FGP is so different we do not have the option of porting their work for core-geonetwork.

jodygarnett commented 3 years ago

We are expecting to work with content list this:

              <gmd:description xsi:type="gmd:PT_FreeText_PropertyType">
                <gco:CharacterString>Supporting Document;HTML;eng,fra</gco:CharacterString>
                <gmd:PT_FreeText>
                  <gmd:textGroup>
                    <gmd:LocalisedCharacterString locale="#fra">Document de soutien;HTML;eng,fra</gmd:LocalisedCharacterString>
                  </gmd:textGroup>
                </gmd:PT_FreeText>
              </gmd:description>

editor:

layout-custom-fields.xsl: editor form xslt

resources dialog:

iso19139/config/associated-panel/default.json: dialog popup for resources
web-ui/src/main/resources/catalog/components/edit/onlinesrc: angular code and html templates for resource panel

addOnlinesrc.html - template for description text area

jodygarnett commented 3 years ago

Multicombiner like other editor components use the internal identifiers assigned to the xml elements by the editor to bind the values.

The dialog does not interact with the metadata editor form, but requests the values required to manage in custom model scope.params. An xslt process in the sever is used to add/update the online resource and save the editor to refresh the page.

Uses gn-multilingual-field directive adding each multi-lingual section in html, binding controls to the scope.params dialog model:

       <textarea data-ng-repeat="(key, val) in mdLangs"
                        id="gn-addonlinesrc-description-multilingual-{{val}}"
                        rows="3"
                        lang="{{val}}"
                        data-ng-required="params.linkType.fields.applicationProfile.required"
                        data-ng-model="params.desc[val]"

Next step is to create a new directive based on Muilticombiner to bind the controls to the dialog model, however having some trouble with the model not getting updated properly.

            <div data-gn-multientry-combiner-online-resources-description="{{params.linkType.fields.desc.directiveConfig}}"
                   class="col-sm-9 col-xs-11 gn-value nopadding-in-table"
                   data-label="description">
                <input type="hidden"
                          id="gn-addonlinesrc-description-single-textarea"
                          data-ng-required="params.linkType.fields.desc.required"
                          lang="{{val}}"
                          data-ng-model="params.desc[val]" />
            </div>

jodygarnett commented 3 years ago

@juanluisrp has been working on this and making good progress, some issues sync with display to figure out.

Expect a PR next week?

akhamdo commented 3 years ago

would it be possible to know the status on this ? @jodygarnett this is part of our Release 2: March 31.

akhamdo commented 3 years ago

@jodygarnett Would it be possible to have an update on this in our meeting later on today? thanks

jodygarnett commented 3 years ago

@ianwallen have you had a chance to look at https://github.com/geonetwork/core-geonetwork/pull/5559 above?

ianwallen commented 3 years ago

@jodygarnett Sorry, did not realize that it was ready as it was in draft. I was also expecting to see a PR on HNAP with the expected configurations being applied.

I will try to have a look at this soon as I get time.

ianwallen commented 3 years ago

@jodygarnett, @juanluisrp
What changes are required on the HNAP? Is it supposed to be based on https://github.com/metadata101/iso19139.ca.HNAP/tree/description_combiner? As you have tested the changes, could you push the branch with the changes to the HNAP repo?

jodygarnett commented 3 years ago

I wanted to ask if this change is always appropriate for hnap?

Or if this is a specific configuration to use GC_Resource_ContentTypes, GC_Resource_Formats, and GC_Resource_Languages ?

ianwallen commented 3 years ago

According to the HNAP documentation that we received from FGP, this is how the field is supposed to be represented so to answer your question - yes - this does seems to be a specific requirement for HNAP. If we were only talking about NAP then it would remain a free form text.

That being said, it would be nice if this was similar to the org selector, where it is mostly a recommendation of values instead of a a hard coded required value. i.e. with the org selector, I could still enter "Acme Inc" if I wanted to for the organization.

juanluisrp commented 3 years ago

I've created this branch with the required configuration https://github.com/GeoCat/iso19139.ca.HNAP/tree/3.10.x-resource-panel

ianwallen commented 3 years ago

@juanluisrp

I cherry picked commit 06d0410270998c3168cb510a453700fc16bda7e5 into our hnap repository https://github.com/GeoCat/iso19139.ca.HNAP/commit/06d0410270998c3168cb510a453700fc16bda7e5

I cherry picked commit 3c4f2f1dba28f3c3c2ae1ddc0bfe356e3773dd5c into our gn repository https://github.com/geonetwork/core-geonetwork/pull/5559/commits/3c4f2f1dba28f3c3c2ae1ddc0bfe356e3773dd5c

I tested and still get the same results. I was expecting a new form for the description but this is what I see.

I tested the browser and I can see that it is returning the configuration

But it does not seem to be applied. Is there supposed to be an change in the layout-custom-fields.xsl ?

juanluisrp commented 3 years ago

Are you using the geonetwork/core-geonetwork#5559 PR?

ianwallen commented 3 years ago

@juanluisrp

5559 does not have HNAP schema and it is based on 3.12.x so I did not use it directly use GeoCat:3.12.x/online-resource-multicombiner branch from PR #5559.

Instead I cherry picked the one commit that you did from PR #5559 into our branch.

I cherry picked commit 3c4f2f1dba28f3c3c2ae1ddc0bfe356e3773dd5c into our gn repository geonetwork/core-geonetwork@3c4f2f1

Note that my branch is based on 3.10.6 which matches the HNAP 3.10.x branch that we are attempting to place the changes in. I'm assuming that your changes are not specific to any 3.12.x changes. If so - please identify the change and I will cherry pick that as well.

ianwallen commented 3 years ago

@juanluisrp I have tried again.

This time I checkout your branch "GeoCat:3.12.x/online-resource-multicombiner"
Then I added the HNAP schema
Then in the HNAP folder - I checkout out your branch "3.10.x-resource-panel"

I got the same results.

I'm I missing some steps???

@jodygarnett, @josegar74, maybe you could try this to see if you get the same results?

juanluisrp commented 3 years ago

This commit was missing in https://github.com/geonetwork/core-geonetwork/pull/5559 (https://github.com/GeoCat/core-geonetwork/commit/139276db9b87b2c3204b200009121c3b0b6d6d48). @ianwallen please add this change to your branch. I've also added https://github.com/GeoCat/iso19139.ca.HNAP/tree/3.12.x-resource-panel branch with a 3.12.x version of HNAP with the default.json config updated. If you want to test something that has both things integrated you can use https://github.com/GeoCat/core-geonetwork/tree/pr-3.12.x/hnap-resources-dialog branch.

ianwallen commented 3 years ago

@juanluisrp I have tested the new changes and it works however there are still some issues.

1 - The selected values is using the text instead of the code. i.e. for the following if I switch my ui to French, it will save "Document de soutien" as the value instead of "Supporting Document" from "resourcecontenttype#Supporting Document"

2 - languages should be a multi-selection option. Would it be difficult to implement this? The values selected should be comma separated. For now I created an option called "English/French" that may work temporary (but a better solution is to have multi-selection box). Due to the first issue it will save "English/French" when it should save the code name of "eng,fra"

3 - The items should be semi-colon separated without space. The values selected should be comma separated. A good value would be "Dataset;FGDB/GDB;eng,fra" But the current setup puts "Supporting Document; FGDB/GDB; English/French". Notice the extra space between the ";" and the next value.

4 - The formatting seems to be missing the indentations. It should be lined up with the other options.

ianwallen commented 3 years ago

@josegar74, @juanluisrp Note that I have created a pull request with the changes. I believe that most of the issues may be within the geonetwork core.

juanluisrp commented 3 years ago

3 - The items should be semi-colon separated without space. The values selected should be comma separated. A good value would be "Dataset;FGDB/GDB;eng,fra" But the current setup puts "Supporting Document; FGDB/GDB; English/French". Notice the extra space between the ";" and the next value.

This can be configured using the directiveConfig.combiner option in config/associated-panel/default.json. https://github.com/metadata101/iso19139.ca.HNAP/blob/d3827e8d2be0b924da225b0f6a875ee11cb0f779/src/main/plugin/iso19139.ca.HNAP/config/associated-panel/default.json#L30

ianwallen commented 3 years ago

@juanluisrp Thank you - I will update the default.json and remove the extra space next to the ";"

Any idea how hard it would be to fix "1 - The selected values is using the text instead of the code."? I believe that is our show stopper at the moment. We would still like the other issues fixed however we can deploy what we have as long as issue "1" is fixed.

josegar74 commented 3 years ago

@ianwallen this is the snippet created, related to item 1:

  <gmd:description xsi:type="gmd:PT_FreeText_PropertyType">
    <gco:CharacterString>Supporting Document; Android Application; English</gco:CharacterString>
    <gmd:PT_FreeText>
      <gmd:textGroup>
        <gmd:LocalisedCharacterString locale="#fra">Document de soutien; Application Android; Anglais</gmd:LocalisedCharacterString>
      </gmd:textGroup>
    </gmd:PT_FreeText>
  </gmd:description>

I think the specification states to store the traslation value, not the key. In other projects it's used like this.

Or are we misunderstanding your comment?

ianwallen commented 3 years ago

@josegar74 You are partially correct.
I believe I caused confusion by using "Supporting Document" as the sample when there are no issues with that field. The issue is with the 3rd field (language). In your example the value is English and Anglais - but in FGP those are the values that are shown in the form - however it stores "eng" in the data

Here is an example from FGP - you can see the saved language.

                     <gmd:description xsi:type="gmd:PT_FreeText_PropertyType">
                        <gco:CharacterString>Supporting Document;PDF;eng</gco:CharacterString>
                        <gmd:PT_FreeText>
                           <gmd:textGroup>
                              <gmd:LocalisedCharacterString xmlns="" locale="#fra">Document de soutien;PDF;eng</gmd:LocalisedCharacterString>
                           </gmd:textGroup>
                        </gmd:PT_FreeText>
                     </gmd:description>

For now, we could the language thesaurus so that it contains "eng" and "fra" for both languages - but this is not ideal.

ianwallen commented 3 years ago

@josegar74 For now, I'm able to make it work by changing the language thesaurus to only use codes. I pushed the changes as this will help it work for now (although not ideal). Once it is able to support using the codes correctly and it supports a multi selection list for selecting the languages then we can update the language thesaurus.

MichelGabriel commented 3 years ago

4 - The formatting seems to be missing the indentations. It should be lined up with the other options.

This is caused by a column that's too wide in file: https://github.com/geonetwork/core-geonetwork/blob/fac64d75cade8aadb8b095dd377c9141f45c1f10/web-ui/src/main/resources/catalog/components/edit/onlinesrc/partials/addOnlinesrc.html#L277

class="col-sm-12 col-xs-12 gn-value nopadding-in-table" should be class="col-sm-9 col-xs-9 gn-value nopadding-in-table"

juanluisrp commented 2 years ago

[x] 1 - The selected values is using the text instead of the code

[ ] 2 - languages should be a multi-selection option

[x] 3 - The items should be semicolon separated without space, values should be comma separated.

[x] 4 - The formatting seems to be missing the indentations. It should be lined up with the other options.

All the issues except 2 has been covered in this issue. A workaround for 2 has been introduced too: add the following list to the thesaurus used for the languages:

eng
fra
eng,fra This allow to select multiple languages without the need of converting the field in a select multiple.

The fix has been implemented in 3.10.x and 3.12.x. @jodygarnett @ianwallen, can the issue be closed?

wangf1122 commented 2 years ago

@ianwallen

Here is what I tested, it looks good according to the previous conversation.

ianwallen commented 2 years ago

Sorry closed this too quickly. Yes the current interface works. But the language selector is currently not user friendly. As demonstrated above, the users are presented with "eng,fra" as a selection while the same interface from FGP allows for the users selection to be English text instead of language codes. It is probably acceptable to have the selections "English", "French", "English/French" but it is not user friendly to have "eng", "fra", "eng,fra"

ianwallen commented 2 years ago

@wangf1122, can you create a PR which changes GC_Resource_Languages.rdf so that the languages in the dropdown lists are text instead of codes and verify that the codes still work properly in both English and French interface.

If that works, and the PR is merged then I believe we can close this issue

wangf1122 commented 2 years ago

Here is the PR: https://github.com/metadata101/iso19139.ca.HNAP/pull/245

The new view looks like:

ianwallen commented 2 years ago

@juanluisrp

I have tested the PR #245 which seem to have the correct values as shown in @wangf1122 comment above (https://github.com/metadata101/iso19139.ca.HNAP/issues/95#issuecomment-951073418).

However the resulting XML does not seem correct.

<gmd:description xsi:type="gmd:PT_FreeText_PropertyType">
    <gco:CharacterString>Supporting Document;Android Application;English,French</gco:CharacterString>
       <gmd:PT_FreeText>
           <gmd:textGroup>
               <gmd:LocalisedCharacterString locale="#fra">Document de soutien;Application Android;Anglais,Français</gmd:LocalisedCharacterString>
          </gmd:textGroup>
     </gmd:PT_FreeText>
</gmd:description>

I was expecting "English,French" to be "eng,fra"

@josegar74 comment above (https://github.com/metadata101/iso19139.ca.HNAP/issues/95#issuecomment-836573281)

I think the specification states to store the traslation value, not the key. In other projects it's used like this.

Based on this "English,French" would be correct - however FGP currently uses "eng,fra".

is the bug in FGP in that it should be using text and not the code? If so can this be fixed in FGP?
is the bug in this app, where there should be an option to store codes instead of text?

jodygarnett commented 1 year ago

@bo-lu are you available to comment on this issue? It is really an application of FGP policy more than anything else.

metadata101 / iso19139.ca.HNAP

resource structured description control to assist filling in multi-lingual content with content type, format and language #95

5559 does not have HNAP schema and it is based on 3.12.x so I did not use it directly use GeoCat:3.12.x/online-resource-multicombiner branch from PR #5559.