Josepeterparker / google-refine

Automatically exported from code.google.com/p/google-refine
Other
0 stars 0 forks source link

Sequential creation of related columns using apply-operation command #500

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?
1.applying add-columns operations using json file 

What is the expected output? What do you see instead?

What version of Google Refine are you using?
2.5
What operating system and browser are you using?
Ubuntu 
Is this problem specific to the type of browser you're using or it happens
in all the browsers you tried?
not browser-specific

Please provide any additional information below.

I am working on Google refine extension that requires applying a number of 
grefine json operations after creating the project .
I implemented a custom apply-operations command  which will be called after 
lunching the /project page.
Basically the json operations are created manually on Google refine then I 
extracted the history into a file that and I am reading in the command doPost 
function and apply them to the project as the following :

public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        Project project = getProject(request);
        String jsonString = getOperationFile();

        try {
            JSONArray a = ParsingUtilities
                    .evaluateJsonStringToArray(jsonString);
            int count = a.length();
            for (int i = 0; i < count; i++) {
                JSONObject obj = a.getJSONObject(i);
                reconstructOperation(project, obj);
            }

            if (project.processManager.hasPending()) {
                respond(response, "{ \"code\" : \"pending\" }");
            } else {
                respond(response, "{ \"code\" : \"ok\" }");
            }
        } catch (JSONException e) {
            respondException(response, e);
        }
    }

The file includes "core/column-addition-by-fetching-urls" operations that 
receive json data , then other operations create columns from the splitting 
that json data in the previously created columns 
The problem that : The function "createProcess" of the  
"ColumnAdditionByFetchingURLsOperation" checks if baseColumnName of the 
operation exists in the project.columnModel before creating it, and throw an 
exception.
That is happening because the creation process of the _baseColumnName is still 
pending or not fully executed. 

public Process createProcess(Project project, Properties options) throws 
Exception {
        Column column = project.columnModel.getColumnByName(_baseColumnName);
        if (column == null) {
            throw new Exception("No column named " + _baseColumnName);
        }
        if (project.columnModel.getColumnByName(_newColumnName) != null) {
            throw new Exception("Another column already named " + _newColumnName);
        }

        Engine engine = createEngine(project);
        engine.initializeFromJSON(_engineConfig);

        Evaluable eval = MetaParser.parse(_urlExpression);

        return new ColumnAdditionByFetchingURLsProcess(
            project, 
            engine,
            eval,
            getBriefDescription(null)
        );
    }

I understand that related columns operations should be executed sequentially as 
their order in the json file.

Original issue reported on code.google.com by gofranshukair@gmail.com on 29 Nov 2011 at 10:14

GoogleCodeExporter commented 8 years ago
Fixed by r2394. Could you please test?

Original comment by dfhu...@gmail.com on 30 Nov 2011 at 11:58

GoogleCodeExporter commented 8 years ago
The bug still exists and will need a non-trivial fix. I'm triaging this issue 
to post 2.5.

Original comment by dfhu...@gmail.com on 3 Dec 2011 at 10:19