search

BUGS-NYU / schedge

API for NYU's course catalog
https://nyu.a1liu.com/api/
MIT License
55 stars 21 forks source link

Scraping Exception #75

Closed Kn99HN closed 4 years ago

Kn99HN commented 4 years ago

Got this error while running ./schedge db scrape --term 1208.

Caused by: org.postgresql.util.PSQLException: A result was returned when none was expected.
        at org.postgresql.jdbc.PgStatement.checkNoResultUpdate(PgStatement.java:258)
        at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:126)
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
        at database.instructors.UpsertInstructor.upsertInstructor(UpsertInstructor.java:27)
        at database.courses.UpdateSections.updateSections(UpdateSections.java:86)
        at actions.ScrapeTerm.lambda$scrapeTerm$2(ScrapeTerm.java:45)
        at database.GetConnection.withConnection(GetConnection.java:35)
A1Liu commented 4 years ago

Try again, this should be fixed on master now.

Kn99HN commented 4 years ago

New error.

va.lang.IllegalArgumentException: type of object is incompatible for object=0.0
        at utils.Utils.setObject(Utils.java:103)
        at utils.Utils.setArray(Utils.java:110)
        at database.courses.UpdateSections.updateSections(UpdateSections.java:105)
        at actions.ScrapeTerm.lambda$scrapeTerm$2(ScrapeTerm.java:45)
        at database.GetConnection.withConnection(GetConnection.java:34)
        at actions.ScrapeTerm.scrapeTerm(ScrapeTerm.java:28)
        at cli.Database.scrape(Database.java:83)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1802)
        at picocli.CommandLine.access$900(CommandLine.java:145)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2141)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2108)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1975)
        at picocli.CommandLine.execute(CommandLine.java:1904)
        at Main.main(Main.java:11)
A1Liu commented 4 years ago

try again?

Kn99HN commented 4 years ago

Got null pointers exception this time. The statement that throws is this one: UPDATE sections SET name = 'Pediatric Dentistry Clinic', name_vec = to_tsvector('CLS09-DN Pediatric Dentistry Clinic'), campus = 'Washington Square', instruction_mode = 'In-Person', min_units = 0.0, location = 2.0, grading = 'TBA', notes = 'Dental Graded', notes_vec = to_tsvector(?), prerequisites = ?, prereqs_vec = to_tsvector(?) WHERE sections.id = ?RETURNING sections.course_id 

java.lang.NullPointerException
        at utils.Utils.setObject(Utils.java:108)
        at utils.Utils.setArray(Utils.java:116)
        at database.courses.UpdateSections.updateSections(UpdateSections.java:105)
        at actions.ScrapeTerm.lambda$scrapeTerm$2(ScrapeTerm.java:45)
        at database.GetConnection.withConnection(GetConnection.java:34)
        at actions.ScrapeTerm.scrapeTerm(ScrapeTerm.java:28)
        at cli.Database.scrape(Database.java:83)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1802)
        at picocli.CommandLine.access$900(CommandLine.java:145)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2141)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2108)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1975)
        at picocli.CommandLine.execute(CommandLine.java:1904)
        at Main.main(Main.java:11)
A1Liu commented 4 years ago

Ok so I fixed a bunch of errors introduced by the previous refactor, just scraped fall 2020 and it worked.

Kn99HN commented 4 years ago

Work like a charm! Thanks