search

pumpitupdev / pumptools

A collection of tools to run Pump It Up games.
The Unlicense
22 stars 7 forks source link

pumpnet doesn't work.. #34

Closed voidderef closed 3 years ago

voidderef commented 3 years ago

In GitLab by @kimmimi on Apr 23, 2021, 18:02

Hello. I'm DEX.

I proceeded to the hook.conf configuration and checked the GET: 200 log and the downloading log in pumptools.log, but the USB profile does not work with the result of "Nonfunctional USB drive".

I think that corrupted data may have been uploaded to the server during the data import and export process.

Please check my data..

My machine ID is [REDACTED].

Thank you.

voidderef commented 3 years ago

In GitLab by @icex2 on Apr 23, 2021, 18:51

changed the description

voidderef commented 3 years ago

In GitLab by @icex2 on Apr 23, 2021, 18:54

Hey,

you have to provide some more info otherwise I cannot tell what's going on. There is also a template you can use when creating an issue, here is a copy-paste of that:

## Summary
<!--- Provide a general summary of the issue in the Title above -->

## Expected behavior
<!--- Tell us what you expected to happen -->

## Current behavior
<!--- Tell us what happened instead of the expected behavior -->

## Detailed Description
<!--- Provide a detailed description of the issue. Include links to screenshots or videos if necessary -->

## Steps to reproduce
<!--- Provide a detailed step by step description how to reproduce this issue -->
1.
2.
3.
4.

## Further things tested
<!--- Test further things to provide more data points for debugging. Remove checkboxes that do not apply to your current bug -->
* [ ] Tested with `ptapi-io-piuio-keyboard.so`
* [ ] Tested with `ptapi-io-piuio-joystick.so`
* [ ] Tested with `ptapi-io-piuio-real.so`
* [ ] Tested with `lib-local.zip` library package
* [ ] Tested with `lib-ld.zip` library package
* [ ] Tested with the following different configuration parameters:
    * "REPLACE ME WITH CONFIG PARAM CHANGED"
    * "REPLACE ME WITH CONFIG PARAM CHANGED" 
    * ...

## Possible solution
<!--- Not obligatory, but suggest a fix/reason for the bug -->

## Context (Environment)
### Pumptools version(s) affected
* <!--- Add one or multiple versions as a bullet list. Get the git hash from the version file as well -->

### Game(s) and version(s) affected
* <!--- Add one or multiple game versions as a bullet list -->

### Log output
See attachment.
<!--- Provide log output as attachment, e.g. pumptools.log and piueb.log. Make sure to censor sensitive data like usernames, home folder paths
or device names if necessary -->

### Configuration files
See attachment.
<!--- Provide any configuration files that you used, e.g. hook.conf. Make sure to censor sensitive data like
paths with user names -->

### Command line arguments
<!--- Provide how you run the game from the command line with any additional arguments that you provided. 
Also, provide any personal or altered scripts that you used. -->

### APIs used
* <!--- List all APIs you used as a bullet list, e.g. piuio-keyboard -->

### OS version
<!--- Provide the version of your distribution and kernel. Use the command uname -a for the latter -->

### Hardware specs
* CPU: <!--- Insert, e.g. Core i7 2600k 3.20ghz -->
* RAM: <!--- Insert, e.g. 16 GB -->
* GPU: <!--- Insert, e.g. Nvidia GeForce GTX 970, 4GB -->
* Controllers/IO: <!--- Insert, e.g. Cobalt Flux over USB -->

Please fill that in and provide the requested data.

Furthermore, I don't see any reasons to have this as a confidential issue. I already removed your machine ID (I stored it in my personal records for later reference). There might also be other people having the same or a similar issue that can provide intput regarding that.

Therefore, I made the issue visible to everyone.

voidderef commented 3 years ago

In GitLab by @icex2 on Apr 23, 2021, 18:54

made the issue visible to everyone

voidderef commented 3 years ago

In GitLab by @kimmimi on Apr 23, 2021, 21:20

Summary

pumpnet doesn't work

Expected behavior

Nonfunctional USB drive result occurs when trying to log in pumpnet.bin

Current behavior

Deploy certs by referring to the guide document, add https address to hook.conf file and specify certs path

Detailed Description

Nonfunctional USB drive result and login failure occurred when attempting to log in with pumpnet.bin file (GMT+9) Attempt to log into pumpnet.bin between 23:30 and 35 minutes based on the time Screenshot_2021-04-23_23-32-10 Screenshot_2021-04-23_23-34-38

When I shutdown the machine after a login failure after rebooting, there was no record of downloading the profile...

Steps to reproduce

  1. Attempting to log in to pumpnet.bin

Further things tested

Possible solution

When communicating to the server, it is believed that the data may have been corrupted during the profile import/export process.

Context (Environment)

Pumptools version(s) affected

pumptools v1.12

Game(s) and version(s) affected

Log output

See attachment. pumptools.log piueb.log

Configuration files

See attachment.

Command line arguments

/pumpos/data/20_nx2/run.sh

APIs used

OS version

Ubuntu 18.04(Custom PUMPOS)

Hardware specs

voidderef commented 3 years ago

In GitLab by @icex2 on Apr 23, 2021, 23:33

The log files do not contain any traces that you started a game and used a USB profile on the USB screen. Maybe you copied the wrong log files? Note that any previous log files are overwritten once you start the game again. So make sure to pull them after you quit the game and before you restart it.

The screenshots are cut off and only show a small section. Therefore, these are not useful. I always need full log files since they contain a lot more information than what is usually shown on screen.

voidderef commented 3 years ago

In GitLab by @kimmimi on Apr 30, 2021, 20:52

I tried logging into pumpnet again today but still failed to log in with "Nonfunctional USB drive" result. A pumptools log is attached, so please check it.

pumptools_20210430_log1.log pumptools_20210430_log2.log

piueb_20210430.log

voidderef commented 3 years ago

In GitLab by @icex2 on May 1, 2021, 20:01

Thanks for the logs. I think I got a bit further with that. Apparently, the game is just fetching the save file and missing out on the rank file. The reason for that is currently unknown. Just documenting my observations so far, still have to come up with any next steps to take that make sense.

Relevant snippet from the log files above:

[M][2021/4/30-18:31:9:24][sec-microdog40][microdog40.c:337] SEC_MICRODOG40_REQ_OPCODE_CONVERT
[M][2021/4/30-18:31:9:24][sec-microdog40][microdog40.c:263] Converting: C9 67 CD E6 AE AF F8 49 18 31 D0 7B CE A2 D3 42  ==> 7534772C
[M][2021/4/30-18:31:9:24][sec-microdog40][microdog40.c:435] Encrypting response...
[M][2021/4/30-18:31:9:24][sec-microdog40][microdog40.c:443] Transaction successful
[M][2021/4/30-18:31:10:691][redir.c][redir.c:385] Redirect /proc/bus/usb/devices -> /sys/kernel/debug/usb/devices
[M][2021/4/30-18:31:10:810][patch-net-profile][net-profile.c:491] open /mnt/0/nx2save.bin 0 0
[I][2021/4/30-18:31:10:810][patch-net-profile][net-profile.c:433] Found pumpnet profile file /mnt/0/pumpnet.bin
[M][2021/4/30-18:31:10:812][patch-net-profile][net-profile.c:372] Setup virtual file, player 0, file_type 0
[M][2021/4/30-18:31:10:812][patch-net-profile][net-profile.c:391] Setup virtual file done, size 30780
[I][2021/4/30-18:31:10:812][patch-net-profile][net-profile.c:460] Profile file player 0, file_type 0 downloading from server...
[I][2021/4/30-18:31:10:812][pumpnet][pumpnet.c:190] [34CDA255C0C6724] Get save
[M][2021/4/30-18:31:10:813][http][http.c:239] [34CDA255C0C6724][https://pumpnet/pumpnet/usbprofile/v1/nx2/save] GET 24 30784
[M][2021/4/30-18:31:12:471][http][http.c:251] [34CDA255C0C6724][https://pumpnet/pumpnet/usbprofile/v1/nx2/save] GET: 200 (41048 32)
[I][2021/4/30-18:31:12:495][patch-net-profile][net-profile.c:477] Downloading file player 0, file_type 0, successful
[M][2021/4/30-18:31:12:495][patch-net-profile][net-profile.c:203] fread /mnt/0/nx2save.bin, pos 0x0, bytes 292
[M][2021/4/30-18:31:12:495][patch-net-profile][net-profile.c:203] fread /mnt/0/nx2save.bin, pos 0x124, bytes 30488
[M][2021/4/30-18:31:12:496][patch-net-profile][net-profile.c:189] fclose /mnt/0/nx2save.bin
[M][2021/4/30-18:31:12:496][patch-net-profile][net-profile.c:402] Destroying virtual file, player 0, file_type 0
[M][2021/4/30-18:31:20:267][redir.c][redir.c:385] Redirect /SCRIPT/UI/SFX_STATION.LUA -> /pumpos/data/20_nx2/game/SCRIPT/UI/SFX_STATION.LUA
[M][2021/4/30-18:31:20:278][sec-microdog40][microdog40.c:426] Decrypting request...
[M][2021/4/30-18:31:20:279][sec-microdog40][microdog40.c:52] Processing request:

Snippet from NXA (flow is identical to NX2) and what I would expect to see also in your log if everything would be fine:

[M][2021/4/7-19:26:12:874][patch-usb-unlock][usb-unlock.c:46] Spoofing serial number of usb stick to kill serial number locking of profile in a moment...
[M][2021/4/7-19:26:12:886][patch-net-profile][net-profile.c:491] open /mnt/0/nxasave.bin 0 0
[I][2021/4/7-19:26:12:886][patch-net-profile][net-profile.c:433] Found pumpnet profile file /mnt/0/pumpnet.bin
[M][2021/4/7-19:26:12:887][patch-net-profile][net-profile.c:372] Setup virtual file, player 0, file_type 0
[M][2021/4/7-19:26:12:887][patch-net-profile][net-profile.c:392] Setup virtual file done, size 90868
[I][2021/4/7-19:26:12:887][patch-net-profile][net-profile.c:463] Profile file player 0, file_type 0 downloading from server...
[I][2021/4/7-19:26:12:887][pumpnet][pumpnet.c:190] [5EA9830027514D1A] Get save
[M][2021/4/7-19:26:12:889][http][http.c:245] [5EA9830027514D1A][https://pumpnet/pumpnet/usbprofile/v1/nxa/save] GET 24 90872
[M][2021/4/7-19:26:13:579][http][http.c:258] [5EA9830027514D1A][https://pumpnet/pumpnet/usbprofile/v1/nxa/save] GET: 200 (121164 32)
[I][2021/4/7-19:26:13:601][patch-net-profile][net-profile.c:480] Downloading file player 0, file_type 0, successful
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:207] fread /mnt/0/nxasave.bin, pos 0x0, bytes 324
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:207] fread /mnt/0/nxasave.bin, pos 0x144, bytes 90544
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:189] fclose /mnt/0/nxasave.bin
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:403] Destroying virtual file, player 0, file_type 0
[M][2021/4/7-19:26:13:602][patch-usb-unlock][usb-unlock.c:69] Caught serial unlock token...unlocking profile
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:491] open /mnt/0/nxarank.bin 0 1
[I][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:433] Found pumpnet profile file /mnt/0/pumpnet.bin
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:372] Setup virtual file, player 0, file_type 1
[M][2021/4/7-19:26:13:602][patch-net-profile][net-profile.c:392] Setup virtual file done, size 24648
[I][2021/4/7-19:26:13:603][patch-net-profile][net-profile.c:463] Profile file player 0, file_type 1 downloading from server...
[I][2021/4/7-19:26:13:603][pumpnet][pumpnet.c:140] [6B16B7A6309D1FF9] Get rank
[M][2021/4/7-19:26:13:603][http][http.c:245] [6B16B7A6309D1FF9][https://pumpnet/pumpnet/usbprofile/v1/nxa/rank] GET 24 24652
[M][2021/4/7-19:26:13:989][http][http.c:258] [6B16B7A6309D1FF9][https://pumpnet/pumpnet/usbprofile/v1/nxa/rank] GET: 200 (32872 32)
[I][2021/4/7-19:26:13:999][patch-net-profile][net-profile.c:480] Downloading file player 0, file_type 1, successful
[M][2021/4/7-19:26:13:999][patch-net-profile][net-profile.c:207] fread /mnt/0/nxarank.bin, pos 0x0, bytes 24648
[M][2021/4/7-19:26:13:999][patch-net-profile][net-profile.c:189] fclose /mnt/0/nxarank.bin
[M][2021/4/7-19:26:13:999][patch-net-profile][net-profile.c:403] Destroying virtual file, player 0, file_type 1
[M][2021/4/7-19:26:13:999][patch-usb-unlock][usb-unlock.c:69] Caught serial unlock token...unlocking profile
voidderef commented 3 years ago

In GitLab by @icex2 on May 1, 2021, 20:05

What I can observe above:

Assumptions:

At that point, I would assume that there is some logic that I have to verify: The game downloads and checks the save data but considers it bad for some reason. Therefore, it does not continue with downloading the rank data.

voidderef commented 3 years ago

In GitLab by @icex2 on May 1, 2021, 20:52

Checking the decompiled code of NX2 154:

No clue why this might be failing here. Maybe some data corruption happened but only on your profile (basically, what you already asumed in your OP). However, I have some direction that I was lacking at the beginning.

voidderef commented 3 years ago

In GitLab by @icex2 on May 1, 2021, 20:53

@kimmimi Did the profile ever successfully load? If not, I will just delete it and create a new one which might already fix the issue.

voidderef commented 3 years ago

In GitLab by @kimmimi on May 2, 2021, 06:14

Still failed to log in. i'm uploading the log, so please check and delete my data on the server. As the offline profile loads normally, it seems that there is a problem with my data on the server. pumptools_20210502.log piueb_20210502.log

voidderef commented 3 years ago

In GitLab by @icex2 on May 3, 2021, 18:42

I created another profile for the same e-mail you already provided. Please use the newly issued credentials that you should have received by e-mail.

voidderef commented 3 years ago

In GitLab by @kimmimi on May 4, 2021, 14:14

I tried loading the newly provided certificate file, machine ID, and pumpnet.bin file, but still failed. Checking the log file, I still can't request the nx2rank.bin file. A related log file is attached, so please check it. [CONF FILE REDACTED] (leaks sensitive information) piueb_20210504.log pumptools_20210504.log

voidderef commented 3 years ago

In GitLab by @icex2 on May 6, 2021, 21:47

I just got around to have some spare time for testing and I tested both your profiles and I can login using them just fine. So pumpnet is fine and it seems like something is wrong with your setup. I still can't spot anything looking out of the ordinary other than the download of the rank data missing in the logs, sorry.

I know this is a bit far-fetched, but can you try:

With both approaches, only setup NX2 by copying the files, unpacking nx2hook from the latest pumptools package and configuring everything for keyboard only and pumpnet.

I am sorry, but that's the best I can come up with right now. =/

voidderef commented 3 years ago

In GitLab by @kimmimi on May 7, 2021, 17:27

With the hardware intact, it was newly created by re-deploy only the PUMPOS and NX2 data back to the new SSD.

After setting pumptools with desktop.cfg and trying to boot, booting was successful, but the keyboard did not work, so I knew that key mapping was necessary, and I ran the piapi-piuio-keyboard-conf file and tried to map the keys.

However, booting nx2 games through run.sh failed after keymapping.

So, when I tried to boot again by setting pumptools with cabinet.cfg and connecting real io, I tried to boot, but I still couldn't log in to pumpnet. When I checked the logs, there was still no log for the nx2rank.bin request. pumptools_20210508.log piueb_20210508.log

If I only need to set up the desktop environment and run the test, can you please explain in detail when and how to proceed with the keyboard mapping?

Also, I don't know if I need to create an issue in pumpos for this problem, but I tried to boot after changing the boot option to ./pumpos.sh conf-boot /mnt/pumpos/pumpos game 20_nx2, but booting failed as a log related to failed was checked.

voidderef commented 3 years ago

In GitLab by @icex2 on May 7, 2021, 19:19

If I only need to set up the desktop environment and run the test, can you please explain in detail when and how to proceed with the keyboard mapping?

I forgot to mention that the keyboard mapping tool piapi-piuio-keyboard-conf needs a desktop environment, e.g. X11, because it uses the keyboard inputs from that. Usually, on a machine you don't need that and I just copied the one I had on my desktop to my machine when I wanted to do some testing without the real IO. It should run just fine once you run it on a normal desktop installation.

Also, I don't know if I need to create an issue in pumpos for this problem, but I tried to boot after changing the boot option to ./pumpos.sh conf-boot /mnt/pumpos/pumpos game 20_nx2, but booting failed as a log related to failed was checked.

If you haven't successfully completed the setup for nx2 and the game doesn't run when you start run.sh of nx2 manually, that's to be expected. The OS just runs the run.sh of NX2 on startup when configured with ./pumpos.sh conf-boot /mnt/pumpos/pumpos game 20_nx2

Again, I need some log output and I might be able to tell you more what's going on.

voidderef commented 3 years ago

In GitLab by @kimmimi on May 9, 2021, 08:10

Tested by redeploying pumpos again with different mk10 hardware and keyboard configuration. However, it still fails to load the nx2rank.bin file. I am attaching the log that I tested, so please check it.. piueb_20210509.log pumptools_20210509.log

voidderef commented 3 years ago

In GitLab by @icex2 on May 9, 2021, 11:21

Ok, I guess potential next steps might be that we have to actually debug this.

One approach might be that I prepare a nx2hook.so file for you that dumps the save file that is downloaded in order to check if the data of the download is actually fine.

However, before we have to put even more effort into that, have you tested this with NXA? The same pumpnet.bin file and machine ID can be used on NXA as well.

voidderef commented 3 years ago

In GitLab by @kimmimi on May 9, 2021, 18:14

I tried to log in to the NXA version with the same desktop environment, but the nxarank.bin file could not be requested. Upload related logs.. piueb_nxa_20210509.log pumptools_nxa_20210509.log

voidderef commented 3 years ago

In GitLab by @icex2 on May 9, 2021, 18:47

Ok, let's go back to nx2 and take this step by step to debug the issue. Please try this nx2hook.so. Once you use your profile with pumpnet, it downloads the file and writes a copy of it to either /tmp/save_dump.bin or /tmp/rank_dump.bin.

Please run, report and also provide the dumped file output.

For reference, wip branch with code changes: https://dev.s-ul.net/hackitup/pumptools/-/tree/bug-pumpnet-rank-not-loading

voidderef commented 3 years ago

In GitLab by @kimmimi on May 9, 2021, 20:25

Replace the provided nx2hook.so file, rebuild pumptools, and proceed with the test. As a result, profile loading was successful. Upload dumped files and logs. rank_dump.bin save_dump.bin

piueb_nx2_20210510.log pumptools_nx2_20210510.log

voidderef commented 3 years ago

In GitLab by @kimmimi on May 9, 2021, 20:29

I want to synchronize my offline profile to pumpnet and use it. Can I see if the corresponding function is provided? Or do I have to start over the data from scratch like now?

voidderef commented 3 years ago

In GitLab by @icex2 on May 9, 2021, 21:19

Replace the provided nx2hook.so file, rebuild pumptools

What do you mean by you "rebuild pumptools"? Can you please elaborate the steps you have done?

As a result, profile loading was successful.

Well, that is somewhat great to hear that it is finally working, but also not because I would not expact that some debug code fixes that problem.

I also just realized that in one of your previous posts the build date is very recent...essentially "yesterday": Pumptools build May 7 2021 23:55:12 gitrev 873c9fb. That does not align with the build date of the most recent public version v1.12. Can you tell EXACTLY which version of pumptools you used and where you downloaded it from? Could it be that you took the source and built it on your local setup on your own? This would be very important for me to know to understand exactly what was going on.

voidderef commented 3 years ago

In GitLab by @icex2 on May 9, 2021, 21:20

Unfortunately, there is currently no such feature available. It is planned once a web front-end is available (which is currently being worked on).

voidderef commented 3 years ago

In GitLab by @kimmimi on May 10, 2021, 04:16

If so, are nx2 and nxa's pumpnet profiles managed separately? Is it possible to transfer data from nx2's pumpnet profile to nxa's pumpnet profile? Or is the data already created?

voidderef commented 3 years ago

In GitLab by @kimmimi on May 10, 2021, 05:30

The following methods are as follows.

  1. Download the PumpTools V1.12 Master source and removed the Build directory and PUMPTOOLS.ZIP files with the MAKE CLEAN command to clear the previously built PUMPTOOLS.
  2. Check the net-profile.c changed file of bug-pumpnet-rank-not-loading. After downloading the existing Net-Profile.c file, you have replaced with Net-Profile.c downloaded and downloaded.
  3. After rebuilding and providing it back to Make Build, download the nx2hook.so file and replaced the build / bin / nx2hook.so file and replaced with the downloaded nx2hook.so file.
  4. Create a PumpTools.zip file with make package and redeploy on NX2
  5. After that, the Pumpnet login attempts to log in and successfully loaded the profile.

and

  1. And with the rebuilt pumptools.zip, I redeployed to nxa and ran the pumpnet login test, but there was a phenomenon that the game freezes at the screen for selecting a mode such as an arcade station.
voidderef commented 3 years ago

In GitLab by @icex2 on May 11, 2021, 19:59

NX2 and NXA profiles are separate on the server, yes. You already have empty profiles created once you signed up for pumpnet.

There is also no option to transfer NX2 to NXA profiles at this point in time. Again, this is a feature that requires a user interface.

voidderef commented 3 years ago

In GitLab by @icex2 on May 11, 2021, 20:02

Why did you build this based on master? I asked you to use the latest publicly available version V1.12 which is not necessarily aligned with master. This is important for me in order know what I am dealing with and hopefully being able to re-produce problems.

If you don't follow the steps I outline, I cannot help you efficiently and it just wastes everyone's time.

voidderef commented 3 years ago

In GitLab by @kimmimi on May 12, 2021, 01:47

Wasn't v1.12 the source currently uploaded to master? I found 1.12 in the tag field. If the master source and the v1.12 source are different, i will get the v1.12 source again and rebuild it and test it. E84BEC79-A5AD-435A-9A68-D4F676A7BA09

voidderef commented 3 years ago

In GitLab by @icex2 on May 20, 2021, 20:43

You should not rebuild on your own because I do not know for sure if the builds are reproducable (which means you get the absolute exact binaries that I am using for testing and comparison). You pointed out in the description that you used v1.12 which let me to assume these are the released binaries. You did not mention that you built this from source until a few posts. This wasted a lot of your and my time. Now, I don't even know how to conclude this issue properly other than closing it because you got it working by one way or the other.

Next time, be very precise about what you have used and what you have done. We might have figured out that issue already after just a few posts and days...