400 (Bad Request) - An error occurred. - when installing and testing from scratch

[hsiddique]

Hi,

I have gone through the steps to install and load up geoportal server for the first time. When the site loads, i get an error in red which reads

I have gone into Dev tools , and I get a 404 and 400 on the following url

http://[localhost:8080/geoportal-2.7.2/elastic/metadata/_search?from=0&size=10](http://localhost:8080/geoportal-2.7.2/elastic/metadata/_search?from=0&size=10)

When I open the url, within the browser, i get the following

All the install files (JDK, Tomcat, and Elastic Search) are within my users folder (as i want to test before I get IT involved)

[hsiddique]

I have also just tried to create a new metadata using the ArcGIS Metadata style, when I click Save i get the following for the same url as above, this time i get

{ "error": { "root_cause": [ { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on [sys_owner_s] in [metadata]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [sys_owner_s] in order to load field data by uninverting the inverted index. Note that this can use significant memory." } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "metadata", "node": "AetRg1NMRz6mMHm1MIdoBw", "reason": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on [sys_owner_s] in [metadata]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [sys_owner_s] in order to load field data by uninverting the inverted index. Note that this can use significant memory." } } ], "caused_by": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on [sys_owner_s] in [metadata]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [sys_owner_s] in order to load field data by uninverting the inverted index. Note that this can use significant memory.", "caused_by": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on [sys_owner_s] in [metadata]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [sys_owner_s] in order to load field data by uninverting the inverted index. Note that this can use significant memory." } } }, "status": 400 }

[mhogeweg]

what version of elasticsearch are you using? and can you check if your elastic is set to use HTTP or HTTPS?

what is the response to http://localhost:9200 or (if HTTPS enabled) https://localhost:9200?

then, assuming either of the two responds, use http://localhost:9200/_cat/indices (or the HTTPS equivalent) to list the indexes currently defined.

[mhogeweg]

Geoportal Server will check for the configured index and create it if needed (typically on first startup). If it cannot connect to the backend, it cannot create the index and further errors will occur

[hsiddique]

what version of elasticsearch are you using? and can you check if your elastic is set to use HTTP or HTTPS?

what is the response to http://localhost:9200 or (if HTTPS enabled) https://localhost:9200?

then, assuming either of the two responds, use http://localhost:9200/_cat/indices (or the HTTPS equivalent) to list the indexes currently defined.

Hi Marten,

I'm using 8.8,2, and using http Below is what I get when I do http://localhost:9200 _"name": "mypc", "cluster_name": "elasticsearch", "cluster_uuid": "IKXh6cHuRJm-UVedBfFlSw", "version": { "number": "8.8.2", "build_flavor": "default", "build_type": "zip", "build_hash": "98e1271edf932a480e4262a471281f1ee295ce6b", "build_date": "2023-06-26T05:16:16.196344851Z", "build_snapshot": false, "lucene_version": "9.6.0", "minimum_wire_compatibility_version": "7.17.0", "minimum_index_compatibilityversion": "7.0.0" }, "tagline": "You Know, for Search" }

and below is what I get when I do http://localhost:9200/_cat/indices _yellow open metadata D5RyQxkISuFmreOrK6PmQ 1 1 5 1 65.7kb 65.7kb

When I launch elasticsearch.bat, the elasticsearch.log is generated, which I have attached

elasticsearch.log

[mhogeweg]

can you run: http://localhost:9200/_mapping ?

since your index is called 'metadata' (the default), you should see something like below. We have seen cases where these dynamic_templates somehow don't get put properly.

    "metadata_v1": {
        "mappings": {
            "dynamic_templates": [
                {
                    "_s": {
                        "match": "*_s",
                        "mapping": {
                        ...

Also curious what errors you're seeing in your geoportal.log.

[hsiddique]

http://localhost:9200/_mapping gives the following

{
    "metadata": {
        "mappings": {
            "properties": {
                "app_editor_s": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "description": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "envelope_cen_pt": {
                    "properties": {
                        "lat": {
                            "type": "float"
                        },
                        "lon": {
                            "type": "float"
                        }
                    }
                },
                "envelope_geo": {
                    "properties": {
                        "coordinates": {
                            "type": "float"
                        },
                        "type": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        }
                    }
                },
                "fileid": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "keywords_s": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "links_s": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "query": {
                    "properties": {
                        "term": {
                            "properties": {
                                "fileid": {
                                    "type": "text",
                                    "fields": {
                                        "keyword": {
                                            "type": "keyword",
                                            "ignore_above": 256
                                        }
                                    }
                                }
                            }
                        }
                    }
                },
                "resources_nst": {
                    "properties": {
                        "url_s": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "url_type_s": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        }
                    }
                },
                "sys_created_dt": {
                    "type": "date"
                },
                "sys_metadatatype_s": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "sys_modified_dt": {
                    "type": "date"
                },
                "sys_owner_s": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "sys_owner_txt": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "sys_xml_clob": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "sys_xmlmeta_obj": {
                    "properties": {
                        "date": {
                            "type": "date"
                        },
                        "hash": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "mimetype": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "size": {
                            "type": "long"
                        }
                    }
                },
                "sys_xmlmodified_dt": {
                    "type": "date"
                },
                "title": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
    }
}

geoportal.log attached geoportal.log

thanks

[mhogeweg]

there was an issue running the 'ensureIndex' function. This function checks if the index is present and tries to create it if needed. Can you make sure elastic is running. Then:

• stop Tomcat
• rename the geoportal server index in app-context.xml (indexName)
• save the file
• restart Tomcat
• wait for a bit and then rerun the requests to list the elastic indexes and the mappings. you should see the dynamic_templates listed for the new index.

[hsiddique]

Made the following change

http://[localhost:9200/_mapping](http://localhost:9200/_mapping) shows the new index, but it is not a dynamic_template

    "metadataaa": {
        "mappings": {
            "properties": {
                "app_editor_s": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                             ......

http://[localhost:9200/_cat/indices](http://localhost:9200/_cat/indices) shows the new indices

yellow open metadata D5RyQxkIS_uFmreOrK6PmQ 1 1 5 1 65.7kb 65.7kb yellow open metadataaa NoXaep8nSkye67ZQSq8LeA 1 1 2 0 16.1kb 16.1kb

But still the same issue as per the first image on my first post

This is in the tomcat log

2024-10-09 10:10:25,316 DEBUG [com.esri.geoportal.lib.elastic.ElasticContextHttp] - Checking index: metadataaa 2024-10-09 10:10:25,377 INFO [com.esri.geoportal.lib.elastic.ElasticContextHttp] - Elasticsearch version: 8.8.2 2024-10-09 10:10:25,382 DEBUG [com.esri.geoportal.lib.elastic.ElasticContextHttp] - Checking index: collections 2024-10-09 10:10:25,384 INFO [com.esri.geoportal.lib.elastic.ElasticContextHttp] - Elasticsearch version: 8.8.2 2024-10-09 10:10:25,386 ERROR [com.esri.geoportal.lib.elastic.ElasticContextHttp] - Error executing ensureIndex() java.lang.NullPointerException: null at java.io.Reader.(Reader.java:168) ~[?:?] at java.io.InputStreamReader.(InputStreamReader.java:108) ~[?:?] at com.esri.geoportal.lib.elastic.http.ElasticClient.send(ElasticClient.java:230) ~[classes/:?] at com.esri.geoportal.lib.elastic.http.ElasticClient.sendHead(ElasticClient.java:269) ~[classes/:?] at com.esri.geoportal.lib.elastic.ElasticContextHttp.ensureIndex(ElasticContextHttp.java:156) [classes/:?] at com.esri.geoportal.lib.elastic.ElasticContextHttp$1.run(ElasticContextHttp.java:265) [classes/:?] at java.util.TimerThread.mainLoop(Timer.java:566) [?:?] at java.util.TimerThread.run(Timer.java:516) [?:?]

[mhogeweg]

looks like it fails on 'ensureIndex' for the collection index. In Geoportal Server, we can now have 2 indexes. One is for the metadata (metadataaa in your case), one is for collections.

in app-context.xml, what is the setting for:

        <beans:property name="supportsCollections" value="false" />

?

[hsiddique]

its the same

changed to true, but no difference.

Just on a side note, I have opened up geoportal on chrome and a sign in box appears for the url http://localhost:8080/geoportal-2.7.2/elastic/metadataaa/_search?from=0&size=10

i've tried all of the following

<security:user name="gptadmin" password="gptadmin" authorities="ROLE_ADMIN,ROLE_PUBLISHER" /> <security:user name="admin" password="admin" authorities="ROLE_ADMIN,ROLE_PUBLISHER" /> <security:user name="publisher" password="publisher" authorities="ROLE_PUBLISHER" /> <security:user name="user" password="user" authorities="ROLE_USER" />

But still get the "An error occured" in red

If i go to http://localhost:8080/geoportal-2.7.2/elastic/metadataaa/_search?from=0&size=10 in a private browser, i get asked to log in, if i do gptadmin or admin, i get the resulting json

{ "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 2, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "metadataaa", "_id": "_search", "_score": 1, "_source": { "query": { "term": { "fileid": "1728464202795r7862281212820916" } } } }, { "_index": "metadataaa", "_id": "733a2deb3d5441dc9aae74480cd7ef96", "_score": 1, "_ignored": [ "sys_xml_clob.keyword" ], "_source": { "fileid": "1728464202795r7862281212820916", "title": "a", "description": "a", "keywords_s": [ "005" ], "app_editor_s": "gxe", "sys_created_dt": "2024-10-09T08:56:57.843Z", "sys_modified_dt": "2024-10-09T08:56:57.843Z", "sys_xmlmodified_dt": "2024-10-09T08:56:57.843Z", "sys_owner_s": "admin", "sys_owner_txt": "admin", "sys_metadatatype_s": "arcgis", "sys_xml_clob": "\u003C?xml version=\"1.0\" encoding=\"UTF-8\"?\u003E\u003Cmetadata xml:lang=\"en\"\u003E\r\n\t\u003CEsri\u003E\r\n\t\t\u003CCreaDate\u003E2024-10-09\u003C/CreaDate\u003E\r\n\t\t\u003CCreaTime\u003E09:56:42.55\u003C/CreaTime\u003E\r\n\t\t\u003CModDate\u003E2024-10-09\u003C/ModDate\u003E\r\n\t\t\u003CModTime\u003E09:56:56.41\u003C/ModTime\u003E\r\n\t\t\u003CPublishStatus\u003Eeditor:esri.dijit.metadata.editor\u003C/PublishStatus\u003E\r\n\t\t\u003CArcGISFormat\u003E1.0\u003C/ArcGISFormat\u003E\r\n\t\t\u003CArcGISstyle\u003EISO 19139 Metadata Implementation Specification GML3.2\u003C/ArcGISstyle\u003E\r\n\t\t\u003CArcGISProfile\u003EISO19139\u003C/ArcGISProfile\u003E\r\n\t\t\u003CMapLyrSync\u003Efalse\u003C/MapLyrSync\u003E\r\n\t\u003C/Esri\u003E\r\n\t\u003CmdFileID\u003E1728464202795r7862281212820916\u003C/mdFileID\u003E\r\n\t\u003CmdChar\u003E\r\n\t\t\u003CCharSetCd value=\"004\"/\u003E\r\n\t\u003C/mdChar\u003E\r\n\t\u003CmdContact\u003E\r\n\t\t\u003CrpOrgName\u003Ea\u003C/rpOrgName\u003E\r\n\t\t\u003Crole\u003E\r\n\t\t\t\u003CRoleCd value=\"007\"/\u003E\r\n\t\t\u003C/role\u003E\r\n\t\u003C/mdContact\u003E\r\n\t\u003CmdDateSt\u003E2024-10-09\u003C/mdDateSt\u003E\r\n\t\u003CmdTimeSt\u003E09:56:42.55\u003C/mdTimeSt\u003E\r\n\t\u003CdataIdInfo\u003E\r\n\t\t\u003CidCitation\u003E\r\n\t\t\t\u003CresTitle\u003Ea\u003C/resTitle\u003E\r\n\t\t\t\u003Cdate\u003E\r\n\t\t\t\t\u003CpubDate\u003E2024-10-09T09:56:49.343+01:00\u003C/pubDate\u003E\r\n\t\t\t\u003C/date\u003E\r\n\t\t\u003C/idCitation\u003E\r\n\t\t\u003CidAbs\u003Ea\u003C/idAbs\u003E\r\n\t\t\u003CdataLang\u003E\r\n\t\t\t\u003ClanguageCode value=\"aka\"/\u003E\r\n\t\t\t\u003CcountryCode value=\"AT\"/\u003E\r\n\t\t\u003C/dataLang\u003E\r\n\t\t\u003CdataChar\u003E\r\n\t\t\t\u003CCharSetCd value=\"004\"/\u003E\r\n\t\t\u003C/dataChar\u003E\r\n\t\t\u003CtpCat\u003E\r\n\t\t\t\u003CTopicCatCd value=\"005\"/\u003E\r\n\t\t\u003C/tpCat\u003E\r\n\t\t\u003CresConst\u003E\r\n\t\t\t\u003CLegConsts\u003E\r\n\t\t\t\t\u003CaccessConsts\u003E\r\n\t\t\t\t\t\u003CRestrictCd value=\"008\"/\u003E\r\n\t\t\t\t\u003C/accessConsts\u003E\r\n\t\t\t\t\u003CothConsts\u003EOther Constraints\u003C/othConsts\u003E\r\n\t\t\t\u003C/LegConsts\u003E\r\n\t\t\u003C/resConst\u003E\r\n\t\t\u003CresConst\u003E\r\n\t\t\t\u003CLegConsts\u003E\r\n\t\t\t\t\u003CuseConsts\u003E\r\n\t\t\t\t\t\u003CRestrictCd value=\"008\"/\u003E\r\n\t\t\t\t\u003C/useConsts\u003E\r\n\t\t\t\t\u003CothConsts\u003EOther Constraints\u003C/othConsts\u003E\r\n\t\t\t\u003C/LegConsts\u003E\r\n\t\t\u003C/resConst\u003E\r\n\t\u003C/dataIdInfo\u003E\r\n\u003C/metadata\u003E", "sys_xmlmeta_obj": { "mimetype": "application/xml", "hash": "1885541612780101974:-9088695965832461604:1522", "date": "2024-10-09T08:56:58.300Z", "size": 1522 } } } ] } }

which is one i tried to manually add within geoportal

[mhogeweg]

could you test with the support collections settings set to true?

[hsiddique]

I did change it, but to no avail.

[mhogeweg]

I think I have reproduced this behavior locally. I'll do more troubleshooting and get back to you

[hsiddique]

ok, thanks!

[mhogeweg]

I have created a branch https://github.com/Esri/geoportal-server-catalog/tree/2.7.2_github559 with a fix for this behavior. Somehow the code was raising an exception when testing for the existence of an index, but then the handling of that (which would be: create the index) happened not where it should have happened.

You'll see changes in:

• geoportal\src\main\java\com\esri\geoportal\lib\elastic\ElasticContextHttp.java
• geoportal\src\main\java\com\esri\geoportal\lib\elastic\ElasticContext.java

If comfortable, please feel free to get that branch and compile it locally. if not, please let me know.

[hsiddique]

i built the branch, but no difference even though I deleted elastic search and reinstalled tomcat. So not entirely sure what's going on. Having said that, i thought id install 2.7.1.3 - and it works, the dynamic_mapping is viewable and can view the created metadata (i do have 2 issues with iso 19115-3, but i'll raise that as different issues).

[ShameeraJayawardana]

do you have any tutorial or document to install geoportal to the local computer with all requirements (elasticsearch) properly? the installation I did was not successful.

[as0050629]

@hsiddique When geoportal index name is changes in app-context.xml, it needs to reflect in security intercept URL in app-security.xml

For example

Cc: @mhogeweg

[as0050629]

@hsiddique Please take code again from https://github.com/Esri/geoportal-server-catalog/tree/2.7.2_github559. I have fixed NullPointerException. But NullPointerException will not cause index error. Index is still created and you were able to see that with http://localhost:9200/_cat/indices.

So this is little tricky why this index is giving 404 in Search.

For troubleshooting, if you would like to use a new name for index (other than metadata), please make sure to change the name in app-security.xml as well as shown in previous comment.

I looked at your elasticsearch.log, it shows that metadata index exists but no other request is shown here. It means requests are not reaching this elastic instance . if request reaching this elastic, you will see log as below:

[2024-10-16T12:35:57,688][INFO ][o.e.c.m.MetadataMappingService] [DELDEVL023] [metadata]/4vAe-ZH8Rqu2Q8zFfIYzgA] update_mapping [_doc]

It is noticed that in some cases you need to update the nodes in app-context.xml in geoportal\WEB-INF\classes\config to the machine name to connect to Elasticsearch,. So please try that as well.

<beans:property name="nodes">
    <beans:list>
        <beans:value>gptsrv12r2</beans:value>
    </beans:list>
</beans:property>

Please share below things:

1. app-context.xml
2. You have shared browser respons of 404 search request, please click on request with 400 response code and copy the response in developer tool.

Cc: @mhogeweg