Feature/sshoc 443 new fields in search response

[tparkola]

Closes https://github.com/SSHOC/sshoc-marketplace-backend/issues/443

[stefanprobst]

some search result in the seed data have an empty string in accessibleAt, i.e. [""]:

curl http://localhost:8080/api/item-search | jq '.items[].accessibleAt'

returns:

[
  "https://gephi.org/"
]
[
  "https://datacatalogue.cessda.eu/"
]
[
  "http://ws.clarin-pl.eu/websty.shtml?en"
]
[
  ""
]
[
  "https://console.cloud.google.com/marketplace/details/city-of-austin/austin-crime"
]
[
  "https://www.stats.com/"
]
null
null
null
[
  ""
]
[
  "https://www.youtube.com/watch?v=2FqM4gKeNO4"
]

is this a data problem or a bug?

[stefanprobst]

also, is the solr update related to this?

[stefanprobst]

manually tested adding a thumbnail and getting it back in the item-search response. works great :+1:

[tparkola]

I think it is a data problem as in the test data I see two entries in the media_links table that have empty URLs (each related to a different item of training material type). However, based on your comments (thanks!), I did some improvements in handling empty lists and null values. It won't change the behaviour much, but the code is better :) I could handle empty entries, e.g. remove them from indexing but then the data would not be consistent, i.e. search(solr)-based response would be different from regular database-based response - which may not be what we want. Solr update does not have anything to do with that in my opinion.

[stefanprobst]

:+1: thanks, LGTM :shipit:!

---

Solr update does not have anything to do with that

was just curious about the reason for https://github.com/SSHOC/sshoc-marketplace-backend/pull/458/commits/32362be31b7106e726c2146de143515e28dfc13e

[tparkola]

was just curious about the reason for 32362be

Oh, right, sorry I did not explain that (and I felt uncomfortable doing two changes in the same task, but I could not resist ;-)). The reason is "just" to remove critical vulnerabilities that are in the older version of solr...

[KlausIllmayer]

@tparkola Here I again have a problem. The deployment on our Kubernetes server was succesful, but on start I get the error:

ERROR e.s.m.c.MarketplaceExceptionHandler.handleServerException - Server Exception
org.springframework.web.servlet.resource.NoResourceFoundException: No static resource swagger-ui/index.html.
    at org.springframework.web.servlet.resource.ResourceHttpRequestHandler.handleRequest(ResourceHttpRequestHandler.java:585)
    at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:52)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)

Later on it says:

ERROR o.s.boot.SpringApplication.reportFailure - Application run failed
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://api-stage-sshomp-solr:8983/solr: ERROR: [doc=15094] unknown field 'accessible_at'
    at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:746)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:263)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
    at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:234)

Is there any advise you can give?

[KlausIllmayer]

Update: I still have the issue with swagger, which disappeared. But this was not the reason, why the server didn't start. It was because of the changes in the Solr scheme, which needs to be manually solved by deleting the content of /var/solr/data - after this and a restart of solr as well as backend-container everything worked again. Need to remember this for production.