Warnings in json2veris

[pelekhs]

Hello again,

When running the initial code:

vcdb.dir <- VCDB_DIR if (interactive()) { # show progress bar if the session is interactive vcdb <- json2veris(dir=vcdb.dir, schema="../VCDB/vcdb-merged.json", progressbar=TRUE) } else { vcdb <- json2veris(dir=vcdb.dir, schema="../VCDB/vcdb-merged.json")
}

1. I keep getting the warnings below:

Warning messages: 1: In json2veris(dir = vcdb.dir, schema = "../VCDB/vcdb-merged.json", : Column[s]: "action.physical.vector.Privileged access", Not found in schema, source file:../VCDB/data/json/validated/003c2a32-a95b-499b-9e9c-ea268df7bf1a.json 2: In json2veris(dir = vcdb.dir, schema = "../VCDB/vcdb-merged.json", : Column[s]: "plus.attack_difficulty_initial.Not Applicable", Not found in schema, source file:../VCDB/data/json/validated/004a9681-f5bb-4c26-9972-97969fcdf2b0.jsonColumn[s]: "plus.attack_difficulty_legacy.Not Applicable", Not found in schema, source file:../VCDB/data/json/validated/004a9681-f5bb-4c26-9972-97969fcdf2b0.jsonColumn[s]: "plus.attack_difficulty_subsequent.Not Applicable", Not found in schema, source file:../VCDB/data/json/validated/004a9681-f5bb-4c26-9972-97969fcdf2b0.json 3: In json2veris(dir = vcdb.dir, schema = "../VCDB/vcdb-merged.json", : Column[s]: "action.social.vector.Web application", Not found in schema, source file:../VCDB/data/json/validated/0E3A7E6B-5650-487F-BF8A-0FA590A87ED9.json

How can I avoid these warnings? Should not be the vcdb-merged.json file be uploaded automatically when someone adds new features to the database? Additionally I doubt that "Not Applicable" should be an allowed enumeration value, given that NA and Unknown fully cover the need to describe missing data.

2. json2veris returns a data.frame (let's say vcdb) whose column vcdb$victim.orgsize.Unknown is the inverse vector of what it should be. More specifically vcdb$victim.orgsize.Unknown becomes True when one of vcdb$victim.orgsize.Small or vcdb$victim.orgsize.Large is True and False when both are False. This is a bit strange and I am not sure if it also affects other columns as well. Some days ago this was not the case. This issue has appeared less than a month ago.

Thanks in advance!

[HitchRait]

Running into same issue. Would be nice to get some guidance on how to deal with it.

[HitchRait]

Warning message in json2veris(vcdb_dir, schema = schema_loc):
“Column[s]:   "action.physical.vector.Privileged access",  was/were found in the incident, but not in the schema, source file:.../VCDB/data/json/validated/003c2a32-a95b-499b-9e9c-ea268df7bf1a.json”
Warning message in json2veris(vcdb_dir, schema = schema_loc):
“Column[s]:   "plus.attack_difficulty_initial.Not Applicable",  was/were found in the incident, but not in the schema, source file:.../VCDB/data/json/validated/004a9681-f5bb-4c26-9972-97969fcdf2b0.jsonColumn[s]:   "plus.attack_difficulty_legacy.Not Applicable",  was/were found in the incident, but not in the schema, source file:.../VCDB/data/json/validated/004a9681-f5bb-4c26-9972-97969fcdf2b0.jsonColumn[s]:   "plus.attack_difficulty_subsequent.Not Applicable",  was/were found in the incident, but not in the schema, source file:.../VCDB/data/json/validated/004a9681-f5bb-4c26-9972-97969fcdf2b0.json”
Rows: 18 Columns: 2
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): cluster, pattern

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Rows: 5 Columns: 2
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): from, to

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Rows: 0 Columns: 2
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (2): from, to

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Warning message in add_patterns.data.table(veris, centroids = verisr:::models_to_centroids(), :
“There are columns in the clusters not in the data: (action.error.variety.Omission). Removing them from clusters so that clustering can proceed, but you should check your data and if these are significant columns, you should apply a veris update function with veris_update_f to properly handle them.  This error is likely due to using a schema newer than 1.3.5.”