JabRef / jabref

Graphical Java application for managing BibTeX and biblatex (.bib) databases
https://devdocs.jabref.org
MIT License
3.48k stars 2.45k forks source link

Import/Export to external database fails for PostgreSQL in 3.0 of jabref #473

Closed fishforbreakfast closed 8 years ago

fishforbreakfast commented 8 years ago

Hi,

Just tried 3.0. The import and export features for PostgreSQL fail with the following messages:

For Export - "Could not export to SQL database for the following reason: This connection has been closed"

For Import - "Could not import from SQL database for the following reason: This ResultSet is closed."

Establishing a connection to the database returns successfully.

Thanks

Fred

koppor commented 8 years ago

Which version of PostgreSQL do you use? I suppose that 2.11.1 worked fine? If not, which was the JabRef version working correctly?

fishforbreakfast commented 8 years ago

Hi Oliver,

Everything works fine in 2.11.1. I am using 9.4.5 of PostgreSQL. Just thought I'd try 3.0 since it is out. I really like the new look by the way. Thank you for the efforts to continually improve JabRef.

Cheers,

Fred

On 12/06/2015 06:57 AM, Oliver Kopp wrote:

Which version of PostgreSQL do you use? I suppose that 2.11.1 worked fine? If not, which was the JabRef version working correctly?

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-162313649.

oscargus commented 8 years ago

I made some changes that may solve this in #476. Not sure how you can get a complete build of it before being merged though...

simonharrer commented 8 years ago

@oscargus if you create the branch on jabref instead of your local repository, your builds will show up on builds.jabref.org

oscargus commented 8 years ago

Ahh, OK! Will have to read up in that!

@simonharrer Now you created a branch, right? (It looked like.)

simonharrer commented 8 years ago

I think @koppor created one.

koppor commented 8 years ago

@fishforbreakfast Please try the build at http://builds.jabref.org/databasefix/

fishforbreakfast commented 8 years ago

Hi,

Just tried your build with exactly the same behavior. ??? I'll keep checking for more updates to test. If there is some debugging that I could do, let me know.

Cheers,

Fred

oscargus commented 8 years ago

OK! I've reversed some more changes. A new build is available at: http://builds.jabref.org/databasefix/

Not sure what else could be done to get more information about exactly where the problem actually is. (The origin is that I introduced a bit too much automatic closing to get rid of code warnings, sorry about that.)

fishforbreakfast commented 8 years ago

Hi,

Import is working. Export will not work with 2.11 series of database schema. It also will not create a new entry/entries in an empty database. The following error occurs:

"Could not export to SQL database for the following reason: Error: more than one row returned by a subquery used as an expression"

From postgresql log:

STATEMENT:  INSERT INTO entries (jabref_eid, entry_types_id, cite_key, abstract, address, annote, author, booktitle, chapter, comment, crossref, ctlalt_stretch_factor, ctldash_repeated_names, ctlmax_names_forced_etal, ctlnames_show_etal, ctluse_alt_spacing, ctluse_article_number, ctluse_paper, ctluse_url, ctluseforced_etal, doi, edition, editor, eid, file, howpublished, institution, journal, journaltitle, key_, keywords, language, location, month, note, number, organization, pages, pdf, pmid, ps, publisher, school, search, series, title, type, url, volume, year, date, __groupsearch, priority, relevance, printed, qualityassured, readstatus, entrytype, timestamp, owner, __search, __markedentry, bibtexkey, ranking, database_id) VALUES ('00000000', (SELECT entry_types_id FROM entry_types WHERE label='standard'), 'Ferguson', NULL, NULL, NULL, 'Jim Fishback', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, '(555) 555-1111', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Fishback', NULL, '1')

When I exported another database a different error was generated. This one looks to need a semicolon to terminate SQL statements for PostgreSQL??

ERROR:  syntax error at or near "UPDATE" at character 1062
STATEMENT:  UPDATE entry_types SET 
        abstract='gen', address='gen', annote='gen', author='req', booktitle='gen', chapter='gen', comment='gen', crossref='gen', ctlalt_stretch_factor='gen', ctldash_repeated_names='gen', ctlmax_names_forced_etal='gen', ctlnames_show_etal='gen', ctluse_alt_spacing='gen', ctluse_article_number='gen', ctluse_paper='gen', ctluse_url='gen', ctluseforced_etal='gen', doi='gen', edition='gen', editor='gen', eid='gen', file='gen', howpublished='gen', institution='gen', journal='req', journaltitle='gen', key_='gen', keywords='gen', language='gen', location='gen', month='opt', note='opt', number='opt', organization='gen', pages='opt', pdf='gen', pmid='gen', ps='gen', publisher='gen', school='gen', search='uti', series='gen', title='req', type='gen', url='gen', volume='opt', year='req', date='gen', __groupsearch='gen', priority='gen', relevance='gen', printed='gen', qualityassured='gen', readstatus='gen', entrytype='gen', timestamp='gen', owner='gen', __search='gen', __markedentry='gen', bibtexkey='gen', ranking='gen' WHERE label='article'UPDATE entry_types SET 
        abstract='gen', address='opt', annote='gen', author='req', booktitle='gen', chapter='gen', comment='gen', crossref='gen', ctlalt_stretch_factor='gen', ctldash_repeated_names='gen', ctlmax_names_forced_etal='gen', ctlnames_show_etal='gen', ctluse_alt_spacing='gen', ctluse_article_number='gen', ctluse_paper='gen', ctluse_url='gen', ctluseforced_etal='gen', doi='gen', edition='opt', editor='req', eid='gen', file='gen', howpublished='gen', institution='gen', journal='gen', journaltitle='gen', key_='gen', keywords='gen', language='gen', location='gen', month='opt', note='opt', number='opt', organization='gen', pages='gen', pdf='gen', pmid='gen', ps='gen', publisher='req', school='gen', search='uti', series='opt', title='req', type='gen', url='gen', volume='opt', year='req', date='gen', __groupsearch='gen', priority='gen', relevance='gen', printed='gen', qualityassured='gen', readstatus='gen', entrytype='gen', timestamp='gen', owner='gen', __search='gen', __markedentry='gen', bibtexkey='gen', ranking='gen' WHERE label='book'

Cheers,

Fred

oscargus commented 8 years ago

Good to hear that import is working!

To the best of my knowledge the database schema have not changed (on purpose). But clearly, there may be other changes leading to this behavior... @JabRef/developers: what additional fields are introduced in 3.0?

I've corrected the missing semicolon (which seems to be an old problem) and in ten minutes a new version should be available.

Thanks for testing!

simonharrer commented 8 years ago

Hm, the schema should not have been changed.

fishforbreakfast commented 8 years ago

Still receiving this error. From PostgreSQL logs:

ERROR:  more than one row returned by a subquery used as an expression
STATEMENT:  INSERT INTO entries (jabref_eid, entry_types_id, cite_key, abstract, address, annote, author, booktitle, chapter, comment, crossref, ctlalt_stretch_factor, ctldash_repeated_names, ctlmax_names_forced_etal, ctlnames_show_etal, ctluse_alt_spacing, ctluse_article_number, ctluse_paper, ctluse_url, ctluseforced_etal, doi, edition, editor, eid, file, howpublished, institution, journal, journaltitle, key_, keywords, language, location, month, note, number, organization, pages, pdf, pmid, ps, publisher, school, search, series, title, type, url, volume, year, date, __groupsearch, priority, relevance, printed, qualityassured, readstatus, entrytype, timestamp, owner, __search, __markedentry, bibtexkey, ranking, database_id) VALUES ('00000004', (SELECT entry_types_id FROM entry_types WHERE label='article'), '2012', NULL, NULL, NULL, ', American Diabetes Association', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Diabetes Care', NULL, NULL, 'Autoimmunity; Diabetes Mellitus, Type 1, diagnosis/immunology/metabolism; Diabetes Mellitus, Type 2, diagnosis/metabolism; Diabetes Mellitus, chemically induced/classification/diagnosis/genetics/immunology/metabolism; Diabetes, Gestational, diagnosis/metabolism; Endocrine System Diseases, complications; Female; Genetic Diseases, Inborn, complications; Glucose Tolerance Test; Hemoglobin A, Glycosylated, metabolism; Humans; Insulin Resistance; Insulin, deficiency/genetics/metabolism/secretion; Insulin-Secreting Cells, metabolism; Male; Pancreatic Diseases, complications; Pregnancy; Risk Factors', 'eng', NULL, 'Jan', NULL, NULL, NULL, 'S64--S71', NULL, '22187472', NULL, NULL, NULL, NULL, NULL, 'Diagnosis and classification of diabetes mellitus.', NULL, NULL, '35 Suppl 1', '2012', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, '2012', NULL, '3')
fishforbreakfast commented 8 years ago

Just tried the SELECT statement within the VALUES() function from above. It returns multiple values. This is the problem, I think. Here is the query output from psql:

psql jabref-300
psql (9.4.5)
Type "help" for help.

jabref-300=# SELECT entry_types_id FROM entry_types WHERE label='article';
 entry_types_id 
----------------
              1
              2
              4
              7
             11
             16
             22
             29
             37
             46
             56
             67
             79
             92
            106
            121
            137
            154
            172
(19 rows)

Should be only one value. The statement needs an additional definition to return a single value.

Cheers,

Fred

fishforbreakfast commented 8 years ago

hello, again...

Here is the output from the select statement using the 2.11 database:

psql jabref-211
psql (9.4.5)
Type "help" for help.

jabref-211=# SELECT entry_types_id FROM entry_types WHERE label='article';
 entry_types_id 
----------------
              1
(1 row)

jabref-211=#

Hope this helps.

Cheers,

Fred

oscargus commented 8 years ago

Thanks! I'm just wondering why there are that many values in entry_types and if that has something to do with the earlier problem leading to that JabRef creates more and more values in entry_types...

I haven't used SQL in almost 20 years, but could you possibly list entry_types completely for the two databases? I'm thinking that may lead to additional hints. SELECT * FROM entry_types; ?

fishforbreakfast commented 8 years ago

Oscar,

I did a select * from entry_types; on that table. There are numerous columns. Too wide to display properly here. I used pgadmin3 to look at the databases instead. Haven't had the time to really look at things in detail though. The problem is that the field entry_types_id is not unique for 3.0. It is for 2.11 and previous. If the label article is to have a unique id then there should only be one entry_types_id for label article. There needs to be more refinement of the insert into entry_types to insure that uniqueness is preserved. Maybe even some keys and indexes.

Cheers,

Fred

On 12/07/2015 01:29 AM, Oscar Gustafsson wrote:

Thanks! I'm just wondering why there are that many values in entry_types and if that has something to do with the earlier problem leading to that JabRef creates more and more values in entry_types...

I haven't used SQL in almost 20 years, but could you possibly list entry_types completely for the two databases? I'm thinking that may lead to additional hints. SELECT * FROM entry_types; ?

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-162437322.

fishforbreakfast commented 8 years ago

Oscar,

See attached. I exported SELECT * FROM entry_types; for both databases. entry_types-2.11.txt entry_types-3.0.txt

Cheers,

Fred

oscargus commented 8 years ago

Thanks! For some reason (maybe the earlier bugs), JabRef tries to add new values to entry_types, leading to quite a few identical entries. It usually checks if the entry_type is there and in that case doesn't add it. Hence, there should only be one entry returned from the query.

One way to go would be to manually delete everything in entry_types (not the table though as I'm not sure how JabRef behaves then). First time it will probably work without problems, the question is what happens the second time...

fishforbreakfast commented 8 years ago

You should be able to let the RDBMS database back end check for redundancy with SQL statement logic, etc. Using an inner join on a separate table containing types and id values is typical. Having everything in one table gets confusing. Let me know if you need more testing.

Cheers,

Fred

On 12/07/2015 11:47 AM, Oscar Gustafsson wrote:

Thanks! For some reason (maybe the earlier bugs), JabRef tries to add new values to entry_types, leading to quite a few identical entries. It usually checks if the entry_type is there and in that case doesn't add it. Hence, there should only be one entry returned from the query.

One way to go would be to manually delete everything in entry_types (not the table though as I'm not sure how JabRef behaves then). First time it will probably work without problems, the question is what happens the second time...

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-162605229.

oscargus commented 8 years ago

I've installed both PostgreSQL and MySQL now and for PostgreSQL entry_types directly contains 190 entries... It should at least be easier to debug it now...

oscargus commented 8 years ago

New version to try in 10 minutes or so. You will have to remove the additional rows in entry_table (DELETE FROM entry_types), but then I think it should work...

(It appears that MySQL has a quite different syntax for conditional inserting/updating, and, hence, one would need to write two different SQL statements for this operation to work as a single statement.)

fishforbreakfast commented 8 years ago

On 12/07/2015 05:39 PM, Oscar Gustafsson wrote:

New version to try in 10 minutes or so. You will have to remove the additional rows in entry_table (DELETE FROM ENTRY_TYPES), but then I think it should work...

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-162706273.

Entry_types_id is fixed but now unescaped characters are giving errors.
Yes, you could use DELETE from entry_types; I just dropped the database and re-created it to make sure that there were no spurious data still lurking. The logs from PostgreSQL show an error that I was having initially when PostgreSQL support was added about 3 or 4 years ago. It was fixed but now is back. Don't know Java at all but I work a lot with PostgreSQL.

here are the logs:

ERROR: syntax error at or near "s" at character 3493 STATEMENT: INSERT INTO entries (jabref_eid, entry_types_id, cite_key, abstract, address, annote, author, booktitle, chapter, comment, crossref, ctlalt_stretch_factor, ctldash_repeated_names, ctlmax_names_forced_etal, ctlnames_show_etal, ctluse_alt_spacing, ctluse_article_number, ctluse_paper, ctluse_url, ctluseforcedetal, doi, edition, editor, eid, file, howpublished, institution, journal, journaltitle, key, keywords, language, location, month, note, number, organization, pages, pdf, pmid, ps, publisher, school, search, series, title, type, url, volume, year, date, groupsearch, priority, relevance, printed, qualityassured, readstatus, entrytype, timestamp, owner, search, __markedentry, bibtexkey, ranking, database_id) VALUES ('00000011', (SELECT entry_types_id FROM entry_types WHERE label='article'), 'Abdelghaffar2009', 'In adolescents with type 1 diabetes, insulin resistance likely plays a role in the deterioration of metabolic control. In type 1 diabetes, addition of metformin to insulin therapy, to improve insulin sensitivity, has been assessed in a few trials involving few patients or in uncontrolled studies of short duration. No systematic reviews are available up to date to summarize the evidence about metformin addition to insulin therapy in adolescents with type 1 diabetes.To assess the effects of metformin added to insulin therapy for type 1 diabetes mellitus in adolescents.We searched The Cochrane Library, MEDLINE and EMBASE. We also searched databases of ongoing trials, reference lists of relevant reviews, and we contacted experts, authors and manufacturers.Any randomised controlled trial (RCT) of at least three months duration of treatment comparing metformin added to insulin therapy versus insulin therapy alone in adolescents with type 1 diabetes was included. Cross-over and quasi-randomised controlled trials were excluded.Two reviewers read all abstracts, assessed quality and extracted data independently. Authors were contacted for missing data.Only two trials (60 participants) investigating the effect of metformin added to insulin therapy for three months in adolescents with poorly controlled type 1 diabetes could be included. Meta-analysis was not possible due to the clinical and methodological heterogeneity of data. Both studies suggested that metformin treatment lowered glycosylated haemoglobin A1c (HbA1c) in adolescents with type 1 diabetes and poor metabolic control. Improvements in insulin sensitivity, body composition or serum lipids were not documented in either study, however, one study showed a decrease in insulin dosage by 10%. Adverse effects were mainly gastrointestinal in both studies and hypoglycaemia in one study. No data on health-related quality of life, all-cause mortality or morbidity are currently available.There is some evidence suggesting improvement of metabolic control in poorly controlled adolescents with type 1 diabetes, on addition of metformin to insulin therapy. Stronger evidence is required from larger studies, carried out over longer time periods to document the long-term effects on metabolic control, health-related quality of life as well as morbidity and mortality in those patients.', NULL, NULL, 'Abdelghaffar, Shereen and Attia, Abdelhamid M.', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, '10.1002/14651858.CD006691.pub2', NULL, NULL, NULL, NULL, NULL, 'Pediatrics, Pediatric Endocrinology and Diabetes, Cairo University, 8/1 El-Nasr St. beside Mc Donald's, New Maadi, Cairo, Cairo, New Maadi, Egypt. sh.abdelghaffar@gmail.com', 'Cochrane Database Syst Rev', NULL, NULL, 'Adolescent; Diabetes Mellitus, Type 1, drug therapy; Drug Therapy, Combination; Humans; Hypoglycemic Agents, administration /&/ dosage; Insulin, administration /&/ dosage; Metformin, administration /&/ dosage; Randomized Controlled Trials as Topic', 'eng', NULL, NULL, NULL, '1', NULL, 'CD006691', NULL, '19160294', NULL, NULL, NULL, NULL, NULL, 'Metformin added to insulin therapy for type 1 diabetes mellitus in adolescents.', NULL, 'http://dx.doi.org/10.1002/14651858.CD006691.pub2', NULL, '2009', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Abdelghaffar2009', NULL, '1'); ERROR: syntax error at or near "{" at character 2519 STATEMENT: INSERT INTO entries (jabref_eid, entry_types_id, cite_key, abstract, address, annote, author, booktitle, chapter, comment, crossref, ctlalt_stretch_factor, ctldash_repeated_names, ctlmax_names_forced_etal, ctlnames_show_etal, ctluse_alt_spacing, ctluse_article_number, ctluse_paper, ctluse_url, ctluseforcedetal, doi, edition, editor, eid, file, howpublished, institution, journal, journaltitle, key, keywords, language, location, month, note, number, organization, pages, pdf, pmid, ps, publisher, school, search, series, title, type, url, volume, year, date, groupsearch, priority, relevance, printed, qualityassured, readstatus, entrytype, timestamp, owner, search, __markedentry, bibtexkey, ranking, database_id) VALUES ('00002527', (SELECT entry_types_id FROM entry_types WHERE label='article'), 'Avila2013', 'It is believed that oxidative stress plays a role in the pathogenesis of diabetes mellitus. Several strategies have been developed with the objective of minimizing diabetic complications. Among these, inhibitors of dipeptidyl peptidase-IV (DPP-IV), which act by blocking degradation of incretin hormones, glucagon-like peptide hormone (GLP-1) and glucose-dependent insulinotropic polypeptide (GIP), have been the focus of many studies. It is known that, among the effects of incretins, we highlight its insulinotropic and cytoprotective effects on pancreatic β-cells. The objective of this study was to evaluate the possible protective effects of treatment with vildagliptin, a DPP-IV inhibitor, in β-cells in an experimental model of type 1 diabetes induced by streptozotocin (STZ).Rats were treated for 4 weeks with vildagliptin at concentrations of 5 and 10 mg/kg. In order to observe the pancreatic damage and the possible protective effects of vildagliptin treatment, we measured stress markers TBARS and protein carbonyl, antioxidant enzymes SOD and catalase, and analyzed pancreatic histology.The treatment was effective in modulating stress in pancreatic tissue, both by reducing levels of stress markers as well as by increasing activity of SOD and catalase. After analyzing the pancreatic histology, we found that vildagliptin was also able to preserve islets and pancreatic β-cells, especially at the concentration of 5 mg/kg.Thus, our results suggest that vildagliptin ameliorates oxidative stress and pancreatic beta cell destruction in type 1 diabetic rats. However, to evaluate the real potential of this medication in type 1 diabetes, further studies are needed.', NULL, NULL, '{\'{A}}vila, Danielle de Lima and Ara{\'{u}}jo, Glaucy Rodrigues de and Silva, Maisa and Miranda, Pedro Henrique de Amorim and Diniz, Mirla Fiuza and Pedrosa, Maria L{\'{u}}cia and Silva, Marcelo Eust{\'{a}}quio and {de Lima}, Wanderson Geraldo and Costa, Daniela Caldeira', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, '10.1016/j.arcmed.2013.03.004', NULL, NULL, NULL, NULL, NULL, 'Núcleo de Pesquisas em Ciências Biológicas-NUPEB, Universidade Federal de Ouro Preto, Ouro Preto, Brazil.', 'Arch Med Res', NULL, NULL, 'Adamantane, analogs /&/ derivatives/pharmacology/therapeutic use; Animals; Antioxidants, metabolism; Biological Markers; Blood Glucose, analysis; Body Weight; Catalase, metabolism; Diabetes Mellitus, Experimental, drug therapy/metabolism/pathology; Diabetes Mellitus, Type 1, drug therapy/metabolism/pathology; Dipeptidyl Peptidase 4, metabolism; Dipeptidyl-Peptidase IV Inhibitors, pharmacology/therapeutic use; Female; Incretins, blood; Insulin, blood; Insulin-Secreting Cells, drug effects/metabolism/pathology; Nitriles, pharmacology/therapeutic use; Oxidation-Reduction; Oxidative Stress, drug effects; Pyrrolidines, pharmacology/therapeutic use; Rats; Streptozocin; Superoxide Dismutase, metabolism', 'eng', NULL, 'Apr', NULL, '3', NULL, '194--202', NULL, '23523961', NULL, NULL, NULL, NULL, NULL, 'Vildagliptin ameliorates oxidative stress and pancreatic beta cell destruction in type 1 diabetic rats.', NULL, 'http://dx.doi.org/10.1016/j.arcmed.2013.03.004', '44', '2013', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, '2015.06.13', 'fred', NULL, NULL, 'Avila2013', NULL, '2')

From what I see, the '{' characters in this particular instance, are not being escaped properly. Looks like there is an "s" that is giving trouble as well. There is also the issue of multiple '\' characters being inserted when a database is imported and then re-exported. I looked at DBExporter.java for both 3.0 and 2.11.1. Couldn't see anything obvious but I will keep looking, at least on the SQL end of things.

There is good documentation for PostgreSQL on inserting special characters as well as a PostgreSQL function to convert text with special characters into a format that the backend will accept. Need to look it up though as I don't have to use it much. Don't know Java at all.

PostgreSQL is enough different from MySQL/MariaDB to need different handling. If you are familiar with Oracle then PostgreSQL is more the same.

Thanks for your help.

Cheers,

Fred

stefan-kolb commented 8 years ago

What is the current status of this issue?

fishforbreakfast commented 8 years ago

On 02/10/2016 05:44 PM, Stefan Kolb wrote:

What is the current status of this issue?

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-182638281.

As far as I know, it still doesn't work (JabRef 3.0-3.2). I am not versed in Java at all but I did some checking on the PostgreSQL end and there is a note on how to handle ' characters which is what my server logs seem to indicate is the problem. I tried to remove all ' in fields but then JabRef goes into some sort of infinite loop and doesn't store the record or any other records. There are no server log errors though. I have to kill JabRef at that point.

Cheers,

Fred

oscargus commented 8 years ago

OK, could you please point to that special character documentation? I guess it is indeed mainly about escaping it in a correct way. My problem is that although I installed both PostgreSQL and MySQL to fix the basics, I have quite limited knowledge of it.

fishforbreakfast commented 8 years ago

On 02/11/2016 09:35 AM, Oscar Gustafsson wrote:

OK, could you please point to that special character documentation? I guess it is indeed mainly about escaping it in a correct way. My problem is that although I installed both PostgreSQL and MySQL to fix the basics, I have quite limited knowledge of it.

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-182919079.

Hi,

Sorry it's taken so long to respond. Things have been a bit out of order here for a stretch. I looked up the documentation for PostgreSQL on escaping ' characters. It looks like either double quoting the string with two single quotes (looks like a double quote in the text here) like this '' or using the typical \' will work. Here is the link to the PostgreSQL documentation:

http://www.postgresql.org/docs/9.5/static/sql-syntax-lexical.html

Also:

http://www.postgresql.org/message-id/CAGFRAbNLH24tzb31YOQO_3ELPesGGb96qL9pY0W-wJYDAF9naQ@mail.gmail.com

I'll try to help with this as I can. Let me know how it goes.

Cheers,

Fred

koppor commented 8 years ago

Any update on this?

fishforbreakfast commented 8 years ago

The only thing that I have been able to determine is that single quotes need to be handled accordingly for PostgreSQL. There seems to be an "endless loop" somewhere in the export code for 3.x that locks up the export process once single quotes ( ' characters are problematic but there is plenty of documentation from the PostgreSQL group on it. ) are formatted properly. I am not proficient in java at all so I have not been able to make any progress on JabRef code. I've been using the 2.11 series for my writing and publishing work in the mean time.

Cheers,

Fred

On 03/08/2016 04:45 AM, Oliver Kopp wrote:

Any update on this?

— Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-193720062.

stefan-kolb commented 8 years ago

@simonharrer Can we reproduce this with your new database tests?

simonharrer commented 8 years ago

@fishforbreakfast can you try out the version available at http://builds.jabref.org/fix-473/ whether it solves your issue? According to my tests, they should.

fishforbreakfast commented 8 years ago

On 04/21/2016 05:05 AM, Simon Harrer wrote:

@fishforbreakfast https://github.com/fishforbreakfast can you try out the version available at http://builds.jabref.org/fix-473/ whether it solves your issue? According to my tests, they should.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-212841648

Looks like at least the database part is working. I tried a particularly large bibliography file to import and export to PostgreSQL and it completed without errors. I have some more that I am going to try. If I find any errors, I will let you know. It may be Monday before I get back to you though.

Cheers,

Fred

simonharrer commented 8 years ago

Great. Waiting for your detailed feedback, tough, before merging the fix in.

fishforbreakfast commented 8 years ago

I'll try to speed things up a little. So far, I've imported and exported two bibtex files to PostgreSQL with no issues except that I had to create a new database for JabRef 3.4. I have to do this with each release of JabRef as tables and such change between versions. Anyway, the import and export references have about 2100 to 2500 entries with lots of special characters in the abstracts, author lists, etc.

Hope this helps!

Fred

On 04/21/2016 12:43 PM, Simon Harrer wrote:

Great. Waiting for your detailed feedback, tough, before merging the fix in.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#issuecomment-213033287

simonharrer commented 8 years ago

Thank you. :)

fishforbreakfast commented 8 years ago

Simon,

Just a followup. Everything is working very well with your fixes so far. Thank you very very much for your efforts and work on JabRef!

Regards,

Fred

On 04/21/2016 02:22 PM, Simon Harrer wrote:

Closed #473 https://github.com/JabRef/jabref/issues/473 via 34494bb https://github.com/JabRef/jabref/commit/34494bbf75bbeeb8d361f79bfcd827b13e5e91eb.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/JabRef/jabref/issues/473#event-636672660

simonharrer commented 8 years ago

Great. :-)

Btw. donations keep us going ;-) http://www.jabref.org/#donations