"Immediately" show dataset after it is uploaded

[larjohn]

You need to create the name of the dataset (here, ?dataset is the URI of the dataset):

CONCAT(REPLACE(str(?dataset), '^.*(#|/)', ""), '__', SUBSTR(MD5(STR(?dataset)),1,5)) AS ?name

Then use this name to call:

soft/[name]

This will reset the parts of the cache needed to include the new dataset.

[badmotor]

Hi @larjohn , I think this issue is still there, could you please confirm? otherwise we need to reopen it... @pierorex @liyakun

[larjohn]

Have you tried the steps above (manually, first)? Is the HTTP sucessful?

[marek-dudas]

Just a report from the user's point-of-view: the issue appears to be still there. The pipeline is triggered, packager correctly announces it finished, the .ttl dump of pipeline output is available, but it takes more than 30 minutes before the dataset appears in explorer (at apps.openbudgets.eu). It seemed alright when I tested last week, but maybe it was just a coincidence or some error at my side that the dataset appeared in the explorer immediately.

[larjohn]

@marek-dudas Have you tried the steps above (manually, first)? Is the HTTP request sucessful?

[marek-dudas]

@larjohn could you please add more info on where to send the request? I can try it, but I don't have the context about Rudolf.

[larjohn]

It will be http://apps.openbudgets.eu/soft/[name]

where [name] stands for the result of the following

SELECT ?name{
CONCAT(REPLACE(str(?dataset_uri), '^.*(#|/)', ""), '__', SUBSTR(MD5(STR(?dataset_uri)),1,5)) AS ?name
}

But you will have to request from @liyakun to update the docker images, as I have added the soft sub-path to nginx configuration a minute ago.

[marek-dudas]

So I accessed http://apps.openbudgets.eu/soft/armenia-test-18-09-2017__bfab7 and first I got a "gateway timeout", after second attempt I got

InvalidArgumentException in SoftCacheClear.php line 46: Model armenia-test-18-09-2017__bfab7 is already cached. No need to rebuild the cache.

But when I go to http://apps.openbudgets.eu/ and search for "Armenia", I don't see the dataset listed (its human-readable name is "Armenia Test 18-09-2017" and I can see a similar dataset from last week marked 14-09-2017). The URI of the newly uploaded dataset is <http://data.openbudgets.eu/resource/dataset/armenia-test-18-09-2017>.

[larjohn]

@marek-dudas can you provide a new dataset to test? This seems to have been removed.

[marek-dudas]

I've just uploaded <http://data.openbudgets.eu/resource/dataset/armenia-rudolf-28-9>

The source csv is here armenia-sample.zip You can use it to re-upload and test as needed. Just open http://apps.openbudgets.eu/packager/provide-data and upload the csv. In the next page, set DataType of column year to date - fiscal year and column executed to Amount. (That is the basic input needed to allow the packager to work.) In the final step, the text you enter into the dataset identifier becomes part of the dataset URI, like this <http://data.openbudgets.eu/resource/dataset/[dataset identifier]>. After the pipeline finishes, the dataset dump will be at http://apps.openbudgets.eu/dumps/fromfdp/[dataset identifier].nt.

[larjohn]

Let's see if the latest commit resolves this...waiting for the server to be refreshed.