victims / victims-web

The victims web application.
GNU Affero General Public License v3.0
8 stars 5 forks source link

Service responding with Status code 400 - Invalid datetime in data #35

Closed abn closed 11 years ago

abn commented 11 years ago

https://victims-websec.rhcloud.com/service/v2/update/2010-01-01T00:00:00/

Returns:

datetime.datetime(2012, 12, 21, 1, 58, 40, 472000) is not JSON serializable
abn commented 11 years ago

RockMongo does not seem to like the load with all the records. :(

abn commented 11 years ago

Assigning to @ashcrow as RockMongo does not seem to load for me.

abn commented 11 years ago

Could this be related to #32 ?

abn commented 11 years ago

Imported data seems to use date "submittedon" value as a string. This might need to be converted to ISODate.

The date in question seems to match the initial seed data in the database.

Query:

{"submittedon": { $gte : ISODate("2012-12-21T01: 58: 40.472Z") }}

Result:

{
  "_id": ObjectId("4fe1117cf80fe23d43c7f290"),
  "_v1": {
    "db_version": 8
  },
  "cves": [
    "CVE-2011-2730"
  ],
  "date": ISODate("2012-12-21T01:44:29.95Z"),
  "db_version": 8,
  "format": "Jar",
  "hash": "1a2ee5525a4525e1df8b7e43e3b95992122ef478990acedb77a345929bc1ca435d614e12202a24fc9f9a038ee9d0a39e5a46b831c49857ca198cf4eafd7b515b",
  "hashes": {
    "sha1": {
      "files": [

      ],
      "combined": ""
    },
    "sha512": {
      "files": [

      ],
      "combined": "1a2ee5525a4525e1df8b7e43e3b95992122ef478990acedb77a345929bc1ca435d614e12202a24fc9f9a038ee9d0a39e5a46b831c49857ca198cf4eafd7b515b"
    }
  },
  "meta": [

  ],
  "name": "spring",
  "status": "RELEASED",
  "submittedon": ISODate("2012-12-21T01:58:40.472Z"),
  "submitter": "ashcrow",
  "vendor": "SpringSource",
  "version": "2.5.6.SEC02"
}
abn commented 11 years ago

@ashcrow Did some digging, could not find anything more that what I have mentioned above. From what I can see this bit of code should handle this condition in service_v2.init.py

        item['date'] = item['date'].isoformat()
        item['submittedon'] = item['submittedon'].isoformat()

Looks like its missing it for some reason. Leaving this one with your for the remainder.

ashcrow commented 11 years ago

On it. I have a feeling there is one or more submittedon or date fields which is not a date in the database.

ashcrow commented 11 years ago

Yeah rockmongo works, but way too slowly :-). I'll see if there is a way I can grant direct Mongo shell access soon.

abn commented 11 years ago

I tried searching for records with submittedon or date of type string. No results showed up.

Besides since the error is thrown by json.dumps() - I wonder if the problem is the item was not correctly converted using isoformat()

ashcrow commented 11 years ago

Found it. It's from a temporary patch I put in yesterday removing one of the datetime to isoformat lines. Removing it shortly.

abn commented 11 years ago

:+1: awesomeness.

ashcrow commented 11 years ago

Patch removed. Tested and it now works. It is slow though which I'll look into. Likely it just needs an index on the date field.