The new Cargo-based wiki system allows us to get much more useful item data much easier. For example, we can get the league restrictions for uniques, we might even automatically find out which uniques are fated or can't drop for other reasons, etc.
All of this information is available as a single big table on the wiki now which contains all item information in the whole game. Unfortunately, due to some oversight in the design of Cargo, it can only return 500 items at a time and does not support an offset parameter in the API. So to get the data, we need to scrape the Special:ViewData page and extract the data with BeautifulSoup.
The new Cargo-based wiki system allows us to get much more useful item data much easier. For example, we can get the league restrictions for uniques, we might even automatically find out which uniques are fated or can't drop for other reasons, etc.
All of this information is available as a single big table on the wiki now which contains all item information in the whole game. Unfortunately, due to some oversight in the design of Cargo, it can only return 500 items at a time and does not support an offset parameter in the API. So to get the data, we need to scrape the
Special:ViewDatapage and extract the data with BeautifulSoup.