NRGI / resourcecontracts.org

Resource Contracts
http://resourcecontracts.org
GNU General Public License v2.0
16 stars 9 forks source link

Is it possible to publish only PDF, metadata, and annotations, without the text file? #386

Closed KaitlinCCSI closed 8 years ago

KaitlinCCSI commented 8 years ago

There is not enough time for all OLC contracts to go through Mechanical Turk before the launch. If we try to publish a contract, though, we have to publish both the PDF and the OCR text, which may have many errors. If we don't publish the contract, and only publish the metadata or annotations, an error message shows up where the contract should be.

We would ideally like to be able to do one or both of the following scenarios: (1) publish the PDF version of a contract, the metadata, and the annotations, without publishing the OCR text (if it hasn't been through Mechanical Turk); (2) publish the PDF version of a contract and the metadata, without publishing annotations or the OCR text (if it hasn't been through Mechanical Turk). Is this possible?

I don't have the ability to categorize something for sprint 9 or flag as a priority, but would be good to have an answer soon so we know how to deal with this in sprint 9. cc @anderspeders @anjesh @samccsi

anjesh commented 8 years ago

Right now we have pdf and text integrated in APIs and also in the UI - meaning you can't have one without the other. To be able to only publish pdf without text, is not possible with the current setup, though that sounds a good approach for later iteration. We can publish one of these three independently - metadata, annotations and pdf-text. As for now, what we could do is to replace the ocr-text (if processed by the system but not proper) with something like "Under processing. please see pdf for now" or similar text, whenever people try to view the text version. In that case we don't have do any change in the current infrastructure.

KaitlinCCSI commented 8 years ago

Okay, let's use the option to replace the OCR-text (if processed by the system but it hasn't yet been completed on Mechanical Turk) with the message: "We are currently processing the contract's PDF file, and a text version is not yet available."

For OLC, can you automatically do this for any contract that has not been completed on Mechanical Turk, or do we need to somehow signal which contracts this message is needed for? Thanks!

anjesh commented 8 years ago

Yes that would be great if you could provide the contracts (with urls) that needs to show such message. We will replace the pdf-text with the message - which will get overridden once Mturk is completed and pushed to the system.

SamCCSI commented 8 years ago

Below is the list of contracts for which we need to replace the pdf text with the message.

@anjesh: just confirming - each time MT completes an entire contract, the MT text will automatically override the message and replace it with the MT'ed text?

Many thanks!

cc @kaitlinccsi


Sime Darby Plantation (Liberia) Inc. - Liberia, 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/679/view#/text

Thunder Bird International_Liberia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1046/view#/text

DRC, Industrie de Transformation de Bois, Contrat de concession forestière N. 005, 4 août 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1154/view#/text

[if last 2 pages cannot be added in time] S&P Energy Solutions Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1039/view#/pdf

Liberia, Akewa Group of Companies, Timber Sale Contract, 21 juillet 2010

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1161/view#/text

Liberia, The Liberia Company, Statement of Understanding, 17 December 1949

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1178/view#/text

South Sudan, Nile Trading & Development, Lease Agreement, 11 March 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1180/view#/text

Firestone Liberia, Inc. - Liberia, 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/681/view#/text

Liberia, Bargor & Bargor Enterprise, Contract to Manage Timber Sale Area, 27 June 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1166/view#/text

Toren Agro Industries_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1047/view#/text

Golden Veroleum (Liberia) Inc. - Liberia, 2010

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/682/view#/text

Heng Yue

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1028/view#/text

Liberia, Buchanan Renewables (Monrovia) Power Inc., Concession Agreement, 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1168/view#/pdf

Liberia, Atlantic Resources Limited, Forest Management Contract, 17 September 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1163/view#/text

Liberia, Morris American Rubber Company, Investment Incentive Contract, 10 August 2007

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1175/view#/text

DRC, Bego Congo, Contrat de concession forestière et avenant N. 1, 24 octobre 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1151/view#/text

DRC, Industrie de Transformation des Bois, Contrat de concession forestière N. 012, 12 août 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1156/view#/text

Liberia, Geblo Logging Inc., Forest Management Contract, 17 September 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1170/view#/text

ADA Commercial Ltd Ag Rice Con 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/107/view#/text

] http://rc-site-stage.elasticbeanstalk.com/olc/publicHorizon Plantations PLC_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1029/view#/text

DRC, SODEFOR, Contrat de Concession Forestière, N. 060/14, 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1133/view#/text

DRC, Forabola, Contrat de concession forestière N. 042, 24 octobre 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1152/view#/text

Liberia, B & V Timber Company, Timber Sale Contract, A-6, 27 June 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1164/view#/text

Liberia, Cavalla Rubber Corporation (Liberia) Inc, Concession Agreement, 21 January 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1169/view#/text

Salala

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/108/view#/text

DRC, Bakri-Bois, Contrat de concession forestière et avenant N. 1 au contrat, 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1126/view#/text

Liberia, Salala Rubber Corporation, Concession, 1 August 1959

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1176/view#/text

DRC, Industrie de Transformation des Bois, Contrat de concession forestière N. 013, 12 août 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1157/view#/text

Liberia, International Consultant Capital, Forest Management Contract, 17 September 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1171/view#/text

Global Agricultural Development (Cambodia) Co Ltd_Cambodia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1025/view#/text

Sun Yeun Corporation A-15_Liberia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1044/view#/text

DRC, Forabola, Contrat de concession forestière N. 064, 10 juillet 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1153/view#/text

DRC, Bego Congo, Contrat de concession forestière et avenant N. 1 au contrat, 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1127/view

DRC, SODEFOR, Contrat de Concession Forestière, N. 061/14, 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1134/view

DRC, La Forestière du Lac, Contrat de concession forestière, 27 avril 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1158/view

Liberia, Liberia Forest Products Incorporated, Investment Agreement, 21 December 2007

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1172/view#/text

Liberia, B & V Timber Company, Timber Sale Contract, A-9, 27 June 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1165/view#/text

Al-Mehdi_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1021/view#/text

Liberia, Sun Yeun Corporation, Timber Sale Contract, TSC Area A-16, 21 July 2010

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1177/view#/text

DRC, Maison NBK Service, Contrat de concession forestière N. 049, 25 avril 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1160/view

Goldtree SL Limited and Goldtree Holdings_Sierra Leone

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1026/view#/text

Maryland Oil Palm Plantation - Liberia, 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/680/view#/text

Tarpeh Timber_Liberia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1045/view#/text

Ruchi Agri_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1038/view#/text

On Tue, Sep 29, 2015 at 7:21 AM, Anjesh notifications@github.com wrote:

Yes that would be great if you could provide the contracts (with urls) that needs to show such message. We will replace the pdf-text with the message - which will get overridden once Mturk is completed and pushed to the system.

— Reply to this email directly or view it on GitHub https://github.com/NRGI/resourcecontracts.org/issues/386#issuecomment-143947823 .

SAM SZOKE-BURKE _Legal Researcher_Columbia Center on Sustainable Investment Columbia Law School - The Earth Institute, Columbia University Level 1, Warren Hall, 410 W116th St. New York, NY 10027 | (212) 854-2635 <212-854-2635> | (646) 630-4123 <646-630-4123> | s.burke@columbia.edu | www.ccsi.columbia.edu | @CCSI_Columbia https://twitter.com/CCSI_Columbia | @samszokeburke http://twitter.com/samszokeburke

anderspeders commented 8 years ago

Thanks @SamCCSI

@anjesh this is related to - and pending how we solve this issue #408

anjesh commented 8 years ago

@SamCCSI correct. Once the MT is completed and pushed, the message will be replaced.

anjesh commented 8 years ago

We will introduce a new option for this, as it appears that this will also be an issue later. There will be option in the admin to tick whether to show the text or not - which will be used in the front-end to hide the text. The reason being some of the text looks ok but might need editing. Replacing with message means we will lose that text. And this will also handle https://github.com/NRGI/resourcecontracts.org/issues/408

SamCCSI commented 8 years ago

Thanks!

On Sep 30, 2015, at 7:00 AM, Anjesh notifications@github.com wrote:

We will introduce a new option for this, as it appears that this will also be an issue later. There will be option in the admin to tick whether to show the text or not - which will be used in the front-end to hide the text. The reason being some of the text looks ok but might need editing. Replacing with message means we will lose that text. And this will also handle #408 https://github.com/NRGI/resourcecontracts.org/issues/408

— Reply to this email directly or view it on GitHub https://github.com/NRGI/resourcecontracts.org/issues/386#issuecomment-144281589 .

anjesh commented 8 years ago

All these contracts text are not displayed now. We have added the option to show and hide the text in admin. Enable these when ready.

image

Sime Darby Plantation (Liberia) Inc. - Liberia, 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/679/view#/text

Thunder Bird International_Liberia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1046/view#/text

DRC, Industrie de Transformation de Bois, Contrat de concession forestière N. 005, 4 août 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1154/view#/text

[if last 2 pages cannot be added in time] S&P Energy Solutions Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1039/view#/pdf

Liberia, Akewa Group of Companies, Timber Sale Contract, 21 juillet 2010

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1161/view#/text

Liberia, The Liberia Company, Statement of Understanding, 17 December 1949

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1178/view#/text

South Sudan, Nile Trading & Development, Lease Agreement, 11 March 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1180/view#/text

Firestone Liberia, Inc. - Liberia, 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/681/view#/text

Liberia, Bargor & Bargor Enterprise, Contract to Manage Timber Sale Area, 27 June 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1166/view#/text

Toren Agro Industries_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1047/view#/text

Golden Veroleum (Liberia) Inc. - Liberia, 2010

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/682/view#/text

Heng Yue

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1028/view#/text

Liberia, Buchanan Renewables (Monrovia) Power Inc., Concession Agreement, 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1168/view#/pdf

Liberia, Atlantic Resources Limited, Forest Management Contract, 17 September 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1163/view#/text

Liberia, Morris American Rubber Company, Investment Incentive Contract, 10 August 2007

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1175/view#/text

DRC, Bego Congo, Contrat de concession forestière et avenant N. 1, 24 octobre 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1151/view#/text

DRC, Industrie de Transformation des Bois, Contrat de concession forestière N. 012, 12 août 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1156/view#/text

Liberia, Geblo Logging Inc., Forest Management Contract, 17 September 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1170/view#/text

ADA Commercial Ltd Ag Rice Con 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/107/view#/text

] http://rc-site-stage.elasticbeanstalk.com/olc/publicHorizon Plantations PLC_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1029/view#/text

DRC, SODEFOR, Contrat de Concession Forestière, N. 060/14, 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1133/view#/text

DRC, Forabola, Contrat de concession forestière N. 042, 24 octobre 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1152/view#/text

Liberia, B & V Timber Company, Timber Sale Contract, A-6, 27 June 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1164/view#/text

Liberia, Cavalla Rubber Corporation (Liberia) Inc, Concession Agreement, 21 January 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1169/view#/text

Salala

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/108/view#/text

DRC, Bakri-Bois, Contrat de concession forestière et avenant N. 1 au contrat, 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1126/view#/text

Liberia, Salala Rubber Corporation, Concession, 1 August 1959

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1176/view#/text

DRC, Industrie de Transformation des Bois, Contrat de concession forestière N. 013, 12 août 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1157/view#/text

Liberia, International Consultant Capital, Forest Management Contract, 17 September 2009

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1171/view#/text

Global Agricultural Development (Cambodia) Co Ltd_Cambodia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1025/view#/text

Sun Yeun Corporation A-15_Liberia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1044/view#/text

DRC, Forabola, Contrat de concession forestière N. 064, 10 juillet 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1153/view#/text

DRC, Bego Congo, Contrat de concession forestière et avenant N. 1 au contrat, 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1127/view

DRC, SODEFOR, Contrat de Concession Forestière, N. 061/14, 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1134/view

DRC, La Forestière du Lac, Contrat de concession forestière, 27 avril 2014

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1158/view

Liberia, Liberia Forest Products Incorporated, Investment Agreement, 21 December 2007

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1172/view#/text

Liberia, B & V Timber Company, Timber Sale Contract, A-9, 27 June 2008

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1165/view#/text

Al-Mehdi_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1021/view#/text

Liberia, Sun Yeun Corporation, Timber Sale Contract, TSC Area A-16, 21 July 2010

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1177/view#/text

DRC, Maison NBK Service, Contrat de concession forestière N. 049, 25 avril 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1160/view

Goldtree SL Limited and Goldtree Holdings_Sierra Leone

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1026/view#/text

Maryland Oil Palm Plantation - Liberia, 2011

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/680/view#/text

Tarpeh Timber_Liberia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1045/view#/text

Ruchi Agri_Ethiopia

http://rc-site-stage.elasticbeanstalk.com/olc/public/contract/1038/view#/text