Custom metadata functionality - appears to be there, but cannot use it

[GoogleCodeExporter]

Our extension requires the user to fill out a form during the import stage, 
which collects metadata about the user's dataset (author, date published, data 
license, update frequency...). We needed to store that metadata alongside the 
actual data on the project page - and after spotting hints of "custom metadata" 
functionality in the source code (ProjectMetadata.java), decided to try to use 
it.

We can see that in ProjectMetadata.java - the write() function explicitly saves 
custom metadata if the "mode" is equal to "save", but we don't seem to have a 
way of setting the 'options' parameter.

Apologies, but I've exceeded my issue attachment storage quota, I'll have to 
paste our two custom commands below instead of attaching them:

------------------------------

GetMetaInformationCommand.java

Use:

/linkedgov/command/get-meta-information?project=123456789&keys=LinkedGov.author,
LinkedGov.datePublished,LinkedGov.updateFrequency

public class GetMetaInformationCommand extends Command {

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

        Project project = getProject(request);
        String rawKeysString = request.getParameter("keys");
        String[] keys = rawKeysString.split(",");

        try {
            metadataJSON(response, ProjectManager.singleton.getProjectMetadata(project.id), keys);
        } catch (IOException e) {
            respondException(response, e);
        } catch (ServletException e) {
            respondException(response, e);
        }
    }

    private static void metadataJSON(HttpServletResponse response, ProjectMetadata metadata, String[] keys)
            throws ServletException, IOException {

        try {
            response.setCharacterEncoding("UTF-8");
            response.setHeader("Content-Type", "application/json");

            JSONWriter writer = new JSONWriter(response.getWriter());
            writer.object();
            writer.key("customMetadata");
            writer.object();

            for (int i = 0; i < keys.length; i++){
                String key = keys[i];       
                Serializable value = metadata.getCustomMetadata(key);
                if(value != null){
                                    writer.key(key);
                                    writer.value(value);                    
                }
            }

            writer.endObject();
            writer.endObject();
        } catch (JSONException e) {
            respondException(response, e);
        }       
    }
}

------------------------------

SaveMetaInformationCommand.java

Use:

/linkedgov/command/save-meta-information?project=123456789&LinkedGov.author=Dan 
Smith&LinkedGov.datePublished=07-09-2011&LinkedGov.updateFrequency=monthly

public class SaveMetaInformationCommand extends Command {

    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

        try {
            ProjectMetadata pm = getProjectMetadata(request);

            Enumeration e = request.getParameterNames();

            while (e.hasMoreElements()) {
                String name = (String)e.nextElement();
                if (!name.equals("project")) {
                    String value = request.getParameter(name);
                    pm.setCustomMetadata(name, value);
                }
            }

            respond(response, "{ \"code\" : \"ok\" }");
        } catch (Exception e) {
            respondException(response, e);
        }
    }
}

------------------------------

We call our "save-meta-information" command just before being redirected to the 
project page - which saves our form data as key-value pairs in metadata.json, 
and we call the "get-meta-information" command once the project page has loaded 
- allowing us to store both the form data and the project data together upon 
export.

Hope that makes sense, and apologies if we have missed something!

Original issue reported on code.google.com by danpaulsmith on 23 Nov 2011 at 12:55

[GoogleCodeExporter]

Version 2386

Original comment by danpaulsmith on 23 Nov 2011 at 2:38

[GoogleCodeExporter]

Dan, could you use the preference store instead? That has a command for saving, 
too.

Original comment by dfhu...@gmail.com on 27 Nov 2011 at 9:19

[GoogleCodeExporter]

Ah yes, I've seen that.

Does that save preferences specific to a single project or just for Refine in 
general? We need the former.

We'll be running a single instance of Refine on a remote server for multiple 
users - so we need to store metadata per dataset - and during "create/import 
project" (on the /index page as opposed to saving the data on the /project 
page).

Original comment by danpaulsmith on 28 Nov 2011 at 11:10

[GoogleCodeExporter]

Each project has a preference store and the whole workspace also has one.

(I'm marking this as WontFix.)

Original comment by dfhu...@gmail.com on 29 Nov 2011 at 8:09

• Changed state: WontFix

[GoogleCodeExporter]

Oh right, how can we get & save to a project's preference store?

Original comment by danpaulsmith on 30 Nov 2011 at 11:08

[GoogleCodeExporter]

http://code.google.com/p/google-refine/source/browse/trunk/main/src/com/google/r
efine/commands/SetPreferenceCommand.java uses a particular project's preference 
store if there is a "project" parameter specifying its ID.

Original comment by dfhu...@gmail.com on 30 Nov 2011 at 5:41

[GoogleCodeExporter]

Ok, I can use that instead now.

As a side note, there's still the issue of accessing the project ID on the 
index page moments before being redirected to the project page.

I'm overriding pollImportJob() for now:

/*
 * Override Refine's 'pollImportJob' function
 * 
 * This is necessesary so that we can capture the project's ID (not the import job ID), 
 * which must be used to access and write to the metadata.json file.
 * 
 * We place our own "LinkedGov.saveMetadata()" function inside the block of 
 * code that is able to see the project ID, just before sending the user to the 
 * "project" page.
 */
/*
 * Store the original 'pollImportJob' before we overwrite Refine's version.
 */
LinkedGov.pollImportJob = Refine.CreateProjectUI.prototype.pollImportJob;

Refine.CreateProjectUI.prototype.pollImportJob = function(start, jobID, 
timerID, checkDone, callback, onError) {

    /*
     * Create our own "callback" function
     */
    lgCallback = function(jobID,job) {

        /*
         * This function is accessed twice by Refine, the first time to 
         * send the user to the "preview" panel when they can modify the 
         * import options, and the second time, with the projectID to the 
         * "project" page.
         * 
         * The second time round is when the projectID is present, so we 
         * intercept it to make a call to save our custom metadata.
         */
        if(typeof job.config.projectID != 'undefined'){
            LinkedGov.saveMetadata(jobID, job.config.projectID, function(jobID, projectID){
                Refine.CreateProjectUI.cancelImportinJob(jobID);
                document.location = "project?project=" + projectID;
            });
        } else {
            callback(jobID,job);
        }
    };

    /*
     * Call the original 'pollImportJob' function that was stored at the end of our new 
     * 'pollImportJob' function.
     */
    LinkedGov.pollImportJob(start, jobID, timerID, checkDone, lgCallback, onError);
}

Original comment by danpaulsmith on 1 Dec 2011 at 10:52