search

ajboni / airwindows-cheatsheet-v1

Cheatsheet for airwindows plugins.
https://airwindowscheatsheet.aboni.dev
GNU General Public License v3.0
16 stars 3 forks source link

Feature/airwindopedia_scraper #30

Closed robbiehinch closed 1 year ago

robbiehinch commented 2 years ago

Not really complete. I managed to scrape airwindopedia and the wordpress site, and build the output into database.generated.js. It superficially looks to be the right format, but isn't working if I copy it over database.js and run the website. I don't know the svelte framework at all or how to debug it :/

I added unit testing with JEST, and a unit test launch config for vscode, but they can also be run if the JEST extension is installed, or execute npm test.

ajboni commented 2 years ago

Thanks a lot for this!! Looking great! I cannot take a very detailed look until next week, but the frontend is failing when trying to set the filters for entries without "Type", added a quick fix so you can run the UI.

  1. I think We should handle unknown types as "Unknown" in the scrape step. And handle empty/undefined props as well
  2. I think we should remove the "USE" property as
  3. Everything looks fine with the generated db except we get an "evergreen" entry.
  4. I get some warnings when I try to run the scrape, not sure if this has an impact or not (Abbreviated log)
[xmldom warning]    unclosed xml attribute 
@#[line:223,col:72]
[xmldom warning]    unclosed xml attribute 
@#[line:223,col:76]
[xmldom warning]    unclosed xml attribute 
@#[line:224,col:88]
[xmldom warning]    unclosed xml attribute 
@#[line:225,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:227,col:76]
[xmldom warning]    unclosed xml attribute 
@#[line:228,col:78]
[xmldom warning]    unclosed xml attribute 
@#[line:229,col:68]
[xmldom warning]    unclosed xml attribute 
@#[line:229,col:72]
[xmldom warning]    unclosed xml attribute 
@#[line:230,col:123]
[xmldom warning]    unclosed xml attribute 
@#[line:231,col:115]
[xmldom warning]    unclosed xml attribute 
@#[line:232,col:133]
[xmldom warning]    unclosed xml attribute 
@#[line:233,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:234,col:100]
[xmldom error]  entity not found:© 
@#[line:489,col:17]
[xmldom error]  element parse error: Error: invalid tagName: 
@#[line:531,col:39]
[xmldom warning]    attribute "async" missed value!! "async" instead2!! 
@#[line:570,col:28]
[xmldom warning]    unclosed xml attribute 
@#[line:570,col:211]
[xmldom warning]    unclosed xml attribute 
@#[line:570,col:329]
[xmldom error]  element parse error: Error: attribute equal must after attrName 
@#[line:570,col:1022]
Got info for Vibrato
[xmldom warning]    unclosed xml attribute 
@#[line:28,col:1]
[xmldom warning]    attribute "crossorigin" missed value!! "crossorigin" instead2!! 
@#[line:30,col:1]
[xmldom error]  entity not found:» 
@#[line:31,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:38,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:39,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:40,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:41,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:42,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:43,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:44,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:45,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:46,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:47,col:1]
[xmldom warning]    attribute "1" missed quot(")!! 
@#[line:54,col:1]
[xmldom warning]    attribute "data-noptimize" missed value!! "data-noptimize" instead!! 
@#[line:160,col:1024]
[xmldom warning]    unclosed xml attribute 
@#[line:188,col:85]
[xmldom warning]    unclosed xml attribute 
@#[line:188,col:320]
[xmldom warning]    unclosed xml attribute 
@#[line:188,col:435]
[xmldom warning]    unclosed xml attribute 
@#[line:188,col:439]
[xmldom warning]    unclosed xml attribute 
@#[line:189,col:25]
[xmldom warning]    unclosed xml attribute 
@#[line:191,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:192,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:193,col:107]
[xmldom warning]    unclosed xml attribute 
@#[line:194,col:114]
[xmldom warning]    unclosed xml attribute 
@#[line:195,col:116]
[xmldom warning]    unclosed xml attribute 
@#[line:196,col:97]
[xmldom warning]    unclosed xml attribute 
@#[line:197,col:114]
[xmldom warning]    unclosed xml attribute 
@#[line:198,col:118]
[xmldom warning]    unclosed xml attribute 
@#[line:199,col:113]
[xmldom warning]    unclosed xml attribute 
@#[line:200,col:115]
[xmldom warning]    unclosed xml attribute 
@#[line:201,col:110]
[xmldom warning]    unclosed xml attribute 
@#[line:202,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:203,col:72]
[xmldom warning]    unclosed xml attribute 
@#[line:203,col:76]
[xmldom warning]    unclosed xml attribute 
@#[line:204,col:88]
[xmldom warning]    unclosed xml attribute 
@#[line:205,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:207,col:76]
[xmldom warning]    unclosed xml attribute 
@#[line:208,col:78]
[xmldom warning]    unclosed xml attribute 
@#[line:209,col:68]
[xmldom warning]    unclosed xml attribute 
@#[line:209,col:72]
[xmldom warning]    unclosed xml attribute 
@#[line:210,col:123]
[xmldom warning]    unclosed xml attribute 
@#[line:211,col:115]
[xmldom warning]    unclosed xml attribute 
@#[line:212,col:133]
[xmldom warning]    unclosed xml attribute 
@#[line:213,col:1]
[xmldom warning]    unclosed xml attribute 
@#[line:214,col:100]
[xmldom error]  entity not found:© 
@#[line:469,col:17]
[xmldom error]  element parse error: Error: invalid tagName: 
@#[line:511,col:39]
[xmldom warning]    attribute "async" missed value!! "async" instead2!! 
@#[line:550,col:28]
[xmldom warning]    unclosed xml attribute 
@#[line:550,col:211]
[xmldom warning]    unclosed xml attribute 
@#[line:550,col:329]
[xmldom error]  element parse error: Error: attribute equal must after attrName 
@#[line:550,col:1022]
Got info for Toybox
^C⏎