Some Wii U directories have no content

[Maschell]

Thanks for the tool! I tried to download the Wii U metadata for the DE region, but it's always running into a panic when it tried to download directory 1090723 (which seems to be empty).

Fetching metadata for directory 1090723 (5 out of 53)
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/main.rs:540:56
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Looks like saveShop doesn't like empty directories?

For a workaround see: https://github.com/mikage-emu/saveShop/issues/2#issuecomment-1455083939

[Maschell]

And directory 1090749 seems throw an error. Added my git diff to the first post

[neobrain]

Thanks for trying out saveShop! Wii U support works "in principle", but it's been a while since I tested, so it's likely some minor fixups are needed for the Rust structures encoding the XML schemata. (As my focus is on 3DS, this isn't something I actively work on, so the attempt at fixing the source is appreciated!)

Here's the somewhat more "rusty" version of your changes:

diff --git src/main.rs src/main.rs
index 8556ee3..3009681 100644
--- src/main.rs
+++ src/main.rs
@@ -535,7 +535,13 @@ async fn handle_directory_content(client: &reqwest::Client, directory_id: &str,
         let mut file = File::create(format!("samurai/{}/{}/directory/paginated/{}%3Foffset%3D{}", locale.region, locale.language, directory_id, offset)).unwrap();
         write!(file, "{}", &resp)?;

-        let doc: DirectoryDocument = quick_xml::de::from_str(&resp).unwrap();
+        let doc: DirectoryDocument = quick_xml::de::from_str(&resp)?;
+
+        if doc.directory.contents.as_ref().map(|c| c.total).unwrap_or(0) == 0 {
+            println!("No contents available");
+            directory_info = Some(doc);
+            break;
+        }

         let contents = doc.directory.contents.as_ref().unwrap();

@@ -916,10 +922,13 @@ async fn fetch_metadata(client: &reqwest::Client, locale: &Locale, args: &Args,
     directory_ids.sort_unstable();
     for (index, directory_id) in directory_ids.iter().enumerate() {
         println!("Fetching metadata for directory {} ({} out of {})", directory_id, index + 1, directory_ids.len());
-        let directory: DirectoryDocument = handle_directory_content(&client, &directory_id, &locale).await?;
+        let directory: DirectoryDocument = match handle_directory_content(&client, &directory_id, &locale).await {
+            Ok(dir) => dir,
+            // NOTE: Wii U directory 1090749 is contained in the listing but returns an error page...
+            Err(err) => { println!("  Failed to parse metadata, skipping ({})", err); continue },
+        };
         let directory = directory.directory;
-        assert!(directory.contents.is_some());
-        for content in directory.contents.unwrap().content {
+        for content in directory.contents.into_iter().map(|c| c.content).flatten() {
             match content.title_or_movie {
                 NodeTitleOrMovie::Title(title) => if !title_ids.contains(&title.id) { title_ids.push(title.id) },
                 NodeTitleOrMovie::Movie(movie) => if !movie_ids.contains(&movie.id) { movie_ids.push(movie.id) },

Let me know if you encounter any other issues!

[James-Money]

I'm curious, how were you able to pull Wii U metadata? Did you need a certain Wii U certificate? My Wii U files for my account in mlc01\sys\title\0005001b\10054000\content\ccerts have .der and .aes file types. Would I need to decrypt those to retrieve Wii U eShop data? This tool has has worked amazingly for 3DS, and I'm glad to see your community support, so I'm curious as to how I might branch off into Wii U.

[neobrain]

@James-Money You can run saveShop with --omit-ninja-contents to skip some data to allow fetching the rest without a certificate. I don't know how to extract the client certificate that's otherwise needed from Wii U.

If you dumped your client certificate from 3DS, you can use the same certificate for Wii U though.

[neobrain]

As a side note: shop_id = 4 may contain more applications than shop_id = 2. I don't know which of number is actually used on the Wii U itself.

[Maschell]

As a side note: shop_id = 4 may contain more applications than shop_id = 2. I don't know which of number is actually used on the Wii U itself.

shop_id = 2 has wup in the image/movie URLs, I'm pretty sure it's the Wii u eshop. shop_id 4 (and 3) is maybe boths shops combined?

And to keep everyone updated: Fetching the meta data for all regions and shop_id 2 works with the patches in this issue!

[neobrain]

Patches are integrated in the latest version. Thanks again for testing!