MobilityData / gbfs-validator

The canonical GBFS validator. Maintained by the GBFS community, facilitated by MobilityData.
https://gbfs-validator.mobilitydata.org/
Apache License 2.0
18 stars 12 forks source link

Validating a 1.1 station_status.json doesn't flag presence of incorrect 'num_bikes_available_types' attribute. #50

Open richtaylor-ito opened 2 years ago

richtaylor-ito commented 2 years ago

When validating a v1.1 station_status.json which uses 'num_bike_available_types' in place of 'vehicles_types_available' (as per the 1.1 schema), the validator doesn't flag this.

An example of this can be seen in Bcycle's feed https://gbfs.bcycle.com/bcycle_lametro/gbfs.json :

{ is_returning: 1, is_renting: 1, is_installed: 1, num_docks_available: 23, num_bikes_available: 8, last_reported: 1635497046, num_bikes_available_types: { electric: 4, smart: 0, classic: 4 }, station_id: "bcycle_lametro_3005" }

richtaylor-ito commented 2 years ago

While GBFS does allow for extensions outside of the spec, extensions should be prefixed with an underscore, which would make it possible to distinguish them from mis-named or mis-used attributes.

isabelle-dr commented 2 years ago

Hi @richtaylor-ito, thanks for opening this issue! It seems like this can be handled with additional properties. Adding the following property in the Json Schemas will flag additional properties.

"additionalProperties": false

But this would flag the extensions outside the spec as well... @PierrickP any thoughts on how to solve this problem?

isabelle-dr commented 2 years ago

After trying it with an online Json Schema Validator, using "additionalProperties": false works. We would need to add it to all properties levels in order to flag additional fields in all levels of the JSON file.

Here is the modified station_status file that flags the additional field in Bcycle's feed:

station_status.json

```json { "$schema": "http://json-schema.org/draft-07/schema", "$id": "https://github.com/NABSA/gbfs/blob/master/gbfs.md#station_statusjson", "description": "Describes the capacity and rental availablility of the station", "type": "object", "properties": { "last_updated": { "description": "Last time the data in the feed was updated in POSIX time.", "type": "integer", "minimum": 1450155600 }, "ttl": { "description": "Number of seconds before the data in the feed will be updated again (0 if the data should always be refreshed).", "type": "integer", "minimum": 0 }, "version": { "description": "GBFS version number to which the feed conforms, according to the versioning framework (added in v1.1).", "type": "string", "const": "1.1" }, "data": { "description": "Array that contains one object per station as defined below.", "type": "object", "properties": { "stations": { "type": "array", "items": { "type": "object", "properties": { "station_id": { "description": "Identifier of a station.", "type": "string" }, "num_bikes_available": { "description": "Number of vehicles of any type physically available for rental at the station.", "type": "number", "minimum": 0 }, "num_bikes_disabled": { "description": "Number of disabled vehicles of any type at the station.", "type": "number", "minimum": 0 }, "num_docks_available": { "description": "Number of functional docks physically at the station.", "type": "number", "minimum": 0 }, "num_docks_disabled": { "description": "Number of empty but disabled docks at the station.", "type": "number", "minimum": 0 }, "is_installed": { "description": "Is the station currently on the street?", "type": "number", "minimum": 0, "maximum": 1 }, "is_renting": { "description": "Is the station currently renting vehicles?", "type": "number", "minimum": 0, "maximum": 1 }, "is_returning": { "description": "Is the station accepting vehicle returns?", "type": "number", "minimum": 0, "maximum": 1 }, "last_reported": { "description": "The last time this station reported its status to the operator's backend.", "type": "number", "minimum": 1450155600 } }, "required": [ "station_id", "num_bikes_available", "num_docks_available", "is_installed", "is_renting", "is_returning", "last_reported" ], "additionalProperties": false } } }, "required": ["stations"], } }, "required": ["last_updated", "ttl", "version", "data"], } ```
PierrickP commented 2 years ago

Hello additionalProperties should work and i think there are a tricks with a regex to only accept underscore prefixed key

richfab commented 8 months ago

The spec

"Field names of extensions SHOULD be prefixed with an underscore (_) character." (reference)

Expected outcome:

The validator should return a WARNING when an additional property does not start with an underscore (_).

Potential solution described above by PierrickP:

  1. Set the additionalProperties schema to false to reject additional properties (reference)
  2. Set the patternProperties schema to {"^_": {}} to accept any property starting with underscore ("_") (reference)
  3. Change the validator code to return a WARNING instead of an ERROR for this type of error.

JSON Schema example with additionalProperties and patternProperties:

station_status.json ``` { "$schema": "http://json-schema.org/draft-07/schema", "$id": "https://github.com/MobilityData/gbfs/blob/v3.0-RC2/gbfs.md#station_statusjson", "description": "Describes the capacity and rental availability of the station", "type": "object", "properties": { "last_updated": { "description": "Last time the data in the feed was updated in RFC3339 format.", "type": "string", "format": "date-time" }, "ttl": { "description": "Number of seconds before the data in the feed will be updated again (0 if the data should always be refreshed).", "type": "integer", "minimum": 0 }, "version": { "description": "GBFS version number to which the feed conforms, according to the versioning framework (added in v1.1).", "type": "string", "enum": [ "3.0-RC", "3.0-RC2", "3.0" ] }, "data": { "description": "Array that contains one object per station as defined below.", "type": "object", "properties": { "stations": { "type": "array", "items": { "type": "object", "properties": { "station_id": { "description": "Identifier of a station.", "type": "string" }, "num_vehicles_available": { "description": "Number of vehicles of any type physically available for rental at the station.", "type": "integer", "minimum": 0 }, "vehicle_types_available": { "description": "Array of objects displaying the total number of each vehicle type at the station (added in v2.1-RC).", "type": "array", "items": { "type": "object", "properties": { "vehicle_type_id": { "description": "The vehicle_type_id of vehicle at the station (added in v2.1-RC).", "type": "string" }, "count": { "description": "A number representing the total amount of this vehicle type at the station (added in v2.1-RC).", "type": "integer", "minimum": 0 } }, "required": ["vehicle_type_id", "count"] } }, "num_vehicles_disabled": { "description": "Number of disabled vehicles of any type at the station.", "type": "integer", "minimum": 0 }, "num_docks_available": { "description": "Number of functional docks physically at the station.", "type": "integer", "minimum": 0 }, "num_docks_disabled": { "description": "Number of empty but disabled docks at the station.", "type": "integer", "minimum": 0 }, "is_installed": { "description": "Is the station currently on the street?", "type": "boolean" }, "is_renting": { "description": "Is the station currently renting vehicles?", "type": "boolean" }, "is_returning": { "description": "Is the station accepting vehicle returns?", "type": "boolean" }, "last_reported": { "description": "The last time this station reported its status to the operator's backend in RFC3339 format.", "type": "string", "format": "date-time" }, "vehicle_docks_available": { "description": "Object displaying available docks by vehicle type (added in v2.1-RC).", "type": "array", "items": { "type": "object", "properties": { "vehicle_type_ids": { "description": "An array of strings where each string represents a vehicle_type_id that is able to use a particular type of dock at the station (added in v2.1-RC).", "type": "array", "items": { "type": "string" } }, "count": { "description": "A number representing the total number of available docks for the defined vehicle type (added in v2.1-RC).", "type": "integer", "minimum": 0 } }, "required": ["vehicle_type_ids", "count"] } } }, "required": [ "station_id", "num_vehicles_available", "is_installed", "is_renting", "is_returning", "last_reported" ], "additionalProperties": false, "patternProperties": {"^_": {}} } } }, "required": ["stations"] } }, "required": ["last_updated", "ttl", "version", "data"] } ```