search

dashingsoft / pyarmor

A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts.
http://pyarmor.dashingsoft.com
Other
3.45k stars 291 forks source link

[BUG] `ERROR could not get docker host machine id` Each time the PC is rebooted, the license expires and reverts to a `traial` license #1446

Closed PINTO0309 closed 1 year ago

PINTO0309 commented 1 year ago

1. Summary

An error is displayed during the initial authentication stage and the license activation fails.

ERROR    could not get docker host machine id

I have isolated and verified the situation in detail and report it here. We recently procured a group license.

2. Environment

Ubuntu 22.04

INFO     Python 3.10.12
INFO     Pyarmor 8.3.4
INFO     Platform linux.x86_64

3. Details

  1. First, I installed pyarmor on my working machine. Not in Docker.
  2. We handed over pyarmor-group-device.7 to the pyarmor license administrator on our team to generate a file pyarmor-device-regfile-60xx.7.zip for the group license authorization.
  3. I have Docker installed on my work PC. However, I am not trying to use pyarmor in a Docker container, so I am installing pyarmor directly on the host PC.

  4. Here, I dare not use the pyarmor authentication file right away, but first list only the NW adapter information and basic information about the hard disk.

    $ ifconfig

br-7482e592306c: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:c8  txqueuelen 0  (Ethernet)

br-786b7a65cac1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:bc  txqueuelen 0  (Ethernet)

br-99174413ea6a: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:bd  txqueuelen 0  (Ethernet)

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:49  txqueuelen 0  (Ethernet)

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet xxx.xxx.11.2  netmask 255.255.255.0  broadcast xxx.xxx.11.255
        inet6 fe80::xxxx:xxxx:xxxx:449e  prefixlen 64  scopeid 0x20<link>
        ether 2c:xx:xx:xx:xx:91  txqueuelen 1000  (Ethernet)

enp3s0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 68:xx:xx:xx:xx:05  txqueuelen 1000  (Ethernet)

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)

$ hdparm -i /dev/sda

/dev/sda:

 Model=ST2000DM008-2FR102, FwRev=0001, SerialNo=xxxxxxxx
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=?16?
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=3907029168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-4,5,6,7
  5. Purposefully reboot the PC without performing the initial authentication.

  6. Again, list the hardware information. As a matter of course, the virtual Mac address of the installed Docker service changes each time the host PC is rebooted, unless special changes are made. By the way, my host PC is not equipped with wireless, so I only use the wired IP address of eno1. 

    $ ifconfig

br-7482e592306c: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:fd  txqueuelen 0  (Ethernet)

br-786b7a65cac1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:a5  txqueuelen 0  (Ethernet)

br-99174413ea6a: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:44  txqueuelen 0  (Ethernet)

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet xxx.xxx.0.1  netmask 255.255.0.0  broadcast xxx.xxx.255.255
        ether 02:xx:xx:xx:xx:4c  txqueuelen 0  (Ethernet)

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet xxx.xxx.11.2  netmask 255.255.255.0  broadcast xxx.xxx.xxx.255
        inet6 fe80::xxxx:xxxx:xxxx:449e  prefixlen 64  scopeid 0x20<link>
        ether 2c:xx:xx:xx:xx:91  txqueuelen 1000  (Ethernet)

enp3s0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 68:xx:xx:xx:xx:05  txqueuelen 1000  (Ethernet)

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0

$ hdparm -i /dev/sda

/dev/sda:

 Model=ST2000DM008-2FR102, FwRev=0001, SerialNo=xxxxxxxx
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=?16?
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=3907029168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-4,5,6,7

  7. Here, the initial certification procedure is performed for the first time. 

    $ pyarmor reg .pyarmor/pyarmor-device-regfile-60xx.7.zip

INFO     Python 3.10.12
INFO     Pyarmor 8.3.4 (trial), 000000, non-profits
INFO     Platform linux.x86_64
INFO     register ".pyarmor/pyarmor-device-regfile-60xx.7.zip"
ERROR    could not get docker host machine id
Traceback (most recent call last):
  File "/home/xxxxx/.local/lib/python3.10/site-packages/pyarmor/cli/register.py", line 194, in _get_docker_hostname
    s.connect((host, port))
socket.gaierror: [Errno -2] Name or service not known
ERROR    could not get docker host machine id
  8. I am not using Docker, nor am I running first time authentication, but I get an error message that refers to an id that I don't know what it refers to, called docker host machine id. Note that the license file generated by the administrator with device id=7 at this point cannot be used for authentication in any way. In other words, before we started using pyarmor, we lost 7 licenses out of a total number of licenses with a limit of 100 deployments. Sacrifice 7 licenses to report this issue.

In other words, as long as I use pyarmor 8.3.4, I seem to lose my license every time I reboot my PC. It is quite fatal.

jondy commented 1 year ago

First run pyarmor -d reg .pyarmor/pyarmor-device-regfile-60xx.7.zip to check log, it will show both machine id in license file and current machine id.

Pyarmor 8.3.x introduces a new machine id which ignore virtual network (for example, docker bridge), it should keep same, please check it by this

python3 -c 'from pyarmor.cli.core import pytransform3
print("machine id", pytransform3.get_hd_info(16))'

And check whether it changes after reboot.

If it really changes, I'll build a debug version to check this issue.

BenjaminYde commented 1 year ago

I have the same issue:

System info:

Before reboot:

Screenshot from 2023-08-24 15-10-28

After reboot:

Screenshot from 2023-08-24 15-05-50

PINTO0309 commented 1 year ago

Thanks.

Obviously the values seem to change from boot to boot. Already, the key has changed so many times that the initial value of the authentication file generated is not known.

jondy commented 1 year ago

Please also report os (it seems all of them are linux), physical device or virtual machine. Do not report if it's docker container.

PINTO0309 commented 1 year ago

What else do you want to know about the OS other than what I initially wrote?

jondy commented 1 year ago

I have build debug version pytransform3.so to track machine info, if machine id is changed after reboot, please download it from

https://pyarmor.dashingsoft.com/downloads/temp/libs/cp3.9/pytransform3.so

If you're using other Python version, replace cp3.9 with corresponding version.

Then run this twice, before reboot and after reboot

python3.9 -c"import pytransform3
print(pytransform3.get_hd_info(16))"

Compare console output, and report which items are changed.

PINTO0309 commented 1 year ago

Okay. I'm out of town and will try the day after tomorrow.

alikaz3mi commented 1 year ago 
deepdev@deepdev-H270-Gaming-3:~$ uname -a
Linux deepdev-H270-Gaming-3 6.2.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

https://pyarmor.dashingsoft.com/downloads/temp/libs/cp3.9/pytransform3.so

This link is invalid. Error 404 not found

jondy commented 1 year ago

This link works in my machine.

PINTO0309 commented 1 year ago

@jondy

I just got home from a 2-day business trip to a distant place and immediately tried to download the file from the URL you provided, but I get an HTTP 404 and can't seem to access it.

https://pyarmor.dashingsoft.com/downloads/temp/libs/cp3.9/pytransform3.so https://pyarmor.dashingsoft.com/downloads/temp/libs/cp3.10/pytransform3.so

image

jondy commented 1 year ago

@alikaz3mi @PINTO0309 Please try it again, I fix the access right error.

PINTO0309 commented 1 year ago

@jondy

  1. The display of the MAC address of eno1 is corrupted.
  2. The order in which disk info information is acquired is broken. (I don't know if the order has anything to do with it.)
  3. Values other than TOKEN are basically consistent.

Left: Before reboot Right: After reboot

image

alikaz3mi commented 1 year ago

Before Reboot:

01:node name       : deepdev-H270-Gaming-3
02:enp6s0          : e0d5^02e41f
03:disk info       : wwn-0x5000000000000000-part1
04:disk info       : ata-860_EVO_250GB_CS_0123459948-part1
05:disk info       : wwn-0x50014ee20b6b2f7e-part1
06:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part1
07:disk info       : wwn-0x50014ee20b6b2f7e-part2
08:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part2
09:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790
10:disk info       : wwn-0x50014ee20b6b2f7e
11:disk info       : ata-860_EVO_250GB_CS_0123459948-part2
12:disk info       : wwn-0x5000000000000000-part2
13:disk info       : wwn-0x5000000000000000
14:disk info       : ata-860_EVO_250GB_CS_0123459948
b'ga8f79979e618d37ccdf2f1148ac5406b'

After reboot:

01:node name       : deepdev-H270-Gaming-3
02:enp6s0          : e0d5^02e41f
03:disk info       : ata-860_EVO_250GB_CS_0123459948-part1
04:disk info       : wwn-0x5000000000000000-part1
05:disk info       : wwn-0x50014ee20b6b2f7e-part1
06:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part1
07:disk info       : wwn-0x50014ee20b6b2f7e-part2
08:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part2
09:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790
10:disk info       : wwn-0x50014ee20b6b2f7e
11:disk info       : ata-860_EVO_250GB_CS_0123459948-part2
12:disk info       : wwn-0x5000000000000000-part2
13:disk info       : ata-860_EVO_250GB_CS_0123459948
14:disk info       : wwn-0x5000000000000000
b'g518eb645dd9d9d4e4ae0967486901032'

It seems that the order of some of the disk infos has changed

jondy commented 1 year ago

@alikaz3mi

Got it, thanks, it will be fixed in next release

zzm88 commented 1 year ago

@alikaz3mi

Got it, thanks, it will be fixed in next release

before next release, how to get around this issue?

jondy commented 1 year ago

If machine id is changed after reboot, there is no workaround.

The fixed version will be released in 24 hours.

alikaz3mi commented 1 year ago

I have updated to the newest version of pyarmor. But I am still getting different machine ids:

01:node name       : deepdev-H270-Gaming-3
02:enp6s0          : e0d5^02e41f
03:disk info       : wwn-0x5000000000000000-part1
04:disk info       : ata-860_EVO_250GB_CS_0123459948-part1
05:disk info       : wwn-0x50014ee20b6b2f7e-part2
06:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part2
07:disk info       : wwn-0x50014ee20b6b2f7e-part1
08:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part1
09:disk info       : wwn-0x50014ee20b6b2f7e
10:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790
11:disk info       : wwn-0x5000000000000000-part2
12:disk info       : ata-860_EVO_250GB_CS_0123459948-part2
13:disk info       : wwn-0x5000000000000000
14:disk info       : ata-860_EVO_250GB_CS_0123459948
b'g23cb61b0b7fd21a527b031b901bf6bb1'

After reboot:

01:node name       : deepdev-H270-Gaming-33
02:enp6s0          : e0�5e02e41f
03:disk info       : ata-860_EVO_250GB_CS_0123459948-part31
04:disk info       : wwn-0x5000000000000000-part31
05:disk info       : wwn-0x50014ee20b6b2f7e-part31
06:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part31
07:disk info       : wwn-0x50014ee20b6b2f7e-part32
08:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part32
09:disk info       : wwn-0x50014ee20b6b2f765
10:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ7930
11:disk info       : wwn-0x5000000000000000-part32
12:disk info       : ata-860_EVO_250GB_CS_0123459948-part32
13:disk info       : ata-860_EVO_250GB_CS_01234599438
14:disk info       : wwn-0x50000000000000030
b'g5a1ac30a15c75c4b08e8d482d8b8e3c6'
BenjaminYde commented 1 year ago

I also tested with 8.3.5 and it also still changes when rebooting

Device info:

jondy commented 1 year ago

@BenjaminYde Could you download debug version pytransform3.so above, and report what changes?

PINTO0309 commented 1 year ago

I tried upgrading the version but the situation did not change. Maybe the next version?

jondy commented 1 year ago

And in Pyarmor 8.3.5, test machine id with get_hd_info(20), not get_hd_info(16).

PINTO0309 commented 1 year ago

Looks good. But you haven't updated your debug .so file, have you?

Review the flow with the license administrator and if all is well, close this issue. Please wait a moment.

jondy commented 1 year ago

But you haven't updated your debug .so file, have you?

No, I really didn't updated it. But it's enough to find the problem.

PINTO0309 commented 1 year ago

Obfuscation succeeded with the Group license still in place after reboot. Thank you very much. I would like to restore the 7 licenses that have expired without being used... If possible...

pyarmor gen make_thresholded_relu.py
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006079, xxx
INFO     Platform linux.x86_64
INFO     search inputs ...
INFO     find script make_thresholded_relu.py
INFO     find 1 top resources
INFO     start to generate runtime files
INFO     target platforms {'linux.x86_64'}
INFO     write dist/pyarmor_runtime_006079/pyarmor_runtime.so
INFO     patch runtime file
INFO     generate runtime files OK
INFO     start to obfuscate scripts
INFO     process resource "make_thresholded_relu"
INFO     obfuscating file make_thresholded_relu.py
INFO     write dist/make_thresholded_relu.py
INFO     obfuscate scripts OK
BenjaminYde commented 1 year ago

I have the same with my licenses... I can use them once but when rebooted they are useless.

alikaz3mi commented 1 year ago

I have tested with get_hd_info(20)

before reboot:

print(pytransform3.get_hd_info(20))"
01:node name       : deepdev-H270-Gaming-33
02:enp6s0          : e0�5e02e41f
03:disk info       : ata-860_EVO_250GB_CS_0123459948-part31
04:disk info       : wwn-0x5000000000000000-part31
05:disk info       : wwn-0x50014ee20b6b2f7e-part31
06:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part31
07:disk info       : wwn-0x50014ee20b6b2f7e-part32
08:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part32
09:disk info       : wwn-0x50014ee20b6b2f765
10:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ7930
11:disk info       : wwn-0x5000000000000000-part32
12:disk info       : ata-860_EVO_250GB_CS_0123459948-part32
13:disk info       : ata-860_EVO_250GB_CS_01234599438
14:disk info       : wwn-0x50000000000000030
b'k5a1ac30a15c75c4b08e8d482d8b8e3c6'

but from pyarmor.cli.core import Pytransform3 gave a different machine id :

machine id is "b'kcf899c8d241d00b8649f38df8b2bbd6b'"

After reboot:

print(pytransform3.get_hd_info(20))"
01:node name       : deepdev-H270-Gaming-33
02:enp6s0          : e0�5e02e41f
03:disk info       : wwn-0x5000000000000000-part31
04:disk info       : ata-860_EVO_250GB_CS_0123459948-part31
05:disk info       : wwn-0x50014ee20b6b2f7e-part32
06:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part32
07:disk info       : wwn-0x50014ee20b6b2f7e-part31
08:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ790-part31
09:disk info       : ata-WDC_WD10EURX-63C57Y0_WD-WCC4J7KCZ7930
10:disk info       : wwn-0x50014ee20b6b2f765
11:disk info       : ata-860_EVO_250GB_CS_0123459948-part32
12:disk info       : wwn-0x5000000000000000-part32
13:disk info       : wwn-0x50000000000000030
14:disk info       : ata-860_EVO_250GB_CS_01234599438
b'k2f44b635562f36892c7a8fb1bf1022f6'

and the python cli:

kcf899c8d241d00b8649f38df8b2bbd6b

Although the value from python cli are equal, I am still unable to use the licensed version of pyarmor.

PINTO0309 commented 1 year ago

@alikaz3mi

This may not help you, but I'm posting it just in case.

I discarded the license file with id=7 that I was using for verification in the very first comment that posted this issue, and verified it with a new license file with id=8 generated using pyarmor 8.3.5, and TOKEN remained fixed after the reboot.

You will probably need to generate a new file using the latest pyarmor 8.3.5, as the specification appears to be that TOKEN is embedded in the file that is first generated on the physical device side. Note that the id already used cannot be reused.

The 7 unused licenses I issued for verification are not available.

PINTO0309 commented 1 year ago

Past licenses remain inactive, but will be closed.

alikaz3mi commented 1 year ago

Why is this issue closed? My problem hasn't been solved yet. @jondy. I have generated and tested pyarmor with new license file.

PINTO0309 commented 1 year ago

I see. Sorry about that.

jondy commented 1 year ago

@alikaz3mi

Could you describe problem with shell commands and console log?

Also describe in which machine (is docker, or pyhysic device) to run shell command

And is system python or other python interperter.

Please run pyarmor -d reg xxx to register Pyarmor.

alikaz3mi commented 1 year ago

I am using the systems default python, my machine is a physical desktop PC. Have you seen the get_hd_info(20) that I reported above? It seems that some of the fields are still changing.

Here's the log for registering the 9th license. 

deepdev@deepdev-H270-Gaming-3:~$ pyarmor -d reg pyarmor-device-regfile-6070.9.zip 
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
INFO     register "pyarmor-device-regfile-6070.9.zip"
DEBUG    extracting license.lic
DEBUG    extracting .pyarmor_capsule.zip
DEBUG    group license for machines: ['tokens/kcf899c8d241d00b8649f38df8b2bbd6b']
DEBUG    got machine id: kcf899c8d241d00b8649f38df8b2bbd6b
DEBUG    extracting tokens/kcf899c8d241d00b8649f38df8b2bbd6b
INFO     This license registration information:

License Type    : pyarmor-group
License No.     : pyarmor-vax-006070
License To      : HEMSCap
License Product : dashboard

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation
jondy commented 1 year ago

@alikaz3mi

This bug is fixed in v8.3.5, the temporary uploaded files are still old.

Please test all by Pyarmor 8.3.5

alikaz3mi commented 1 year ago 
ERROR    invalid license token, try to run `pyarmor reg` to register license again
deepdev@deepdev-H270-Gaming-3:~/PycharmProjects/dashboard_project/dashboard_setup$ cat pyarmor.error.log ^C
deepdev@deepdev-H270-Gaming-3:~/PycharmProjects/dashboard_project/dashboard_setup$ pyarmor -v
Pyarmor 8.3.5 (group), 006070, dashboard

License Type    : pyarmor-group
License No.     : pyarmor-vax-006070
License To      : HEMSCap
License Product : dashboard

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation

deepdev@deepdev-H270-Gaming-3:~/PycharmProjects/dashboard_project/dashboard_setup$ cat pyarmor.error.log
1715429 MainProcess 2023-09-03 08:20:01,195
Traceback (most recent call last):
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__main__.py", line 702, in main
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__main__.py", line 690, in main_entry
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__main__.py", line 229, in cmd_gen
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/generate.py", line 156, in process
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/generate.py", line 119, in _obfuscate_scripts
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/core/__init__.py", line 103, in generate_obfuscated_script
  File "<maker>", line 665, in generate_obfuscated_script
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__init__.py", line 16, in process
  File "<maker>", line 504, in process
  File "<maker>", line 510, in coserialize
  File "<maker>", line 555, in _build_ast_body
RuntimeError: invalid license token, try to run `pyarmor reg` to register license again
jondy commented 1 year ago

@alikaz3mi

Could you test the following command with same pyarmor?

pyarmor -d reg pyarmor-group-device-9.zip
touch foo.py
pyarmor -d gen --enable-bcc foo.py

Please provide full commands and full log.

And I'm curious why you just provide only error log but not provide full commands to reproduce the issue as I have told you in previous comments.

And why not try pyarmor reg again just like error message suggestion? Eevn it failed, also report error message which could help fix the problem and save both your time and my time.

It could not help to fix the problem only say pyarmor doesn't work but not provide enough information.

alikaz3mi commented 1 year ago

I'd try to do mo best for solving this issue

  1. Registration
deepdev@deepdev-H270-Gaming-3:~$ pyarmor -d reg pyarmor-device-regfile-6070.10.zip 
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
INFO     register "pyarmor-device-regfile-6070.10.zip"
DEBUG    extracting license.lic
DEBUG    extracting .pyarmor_capsule.zip
DEBUG    group license for machines: ['tokens/kcf899c8d241d00b8649f38df8b2bbd6b']
DEBUG    got machine id: kcf899c8d241d00b8649f38df8b2bbd6b
DEBUG    extracting tokens/kcf899c8d241d00b8649f38df8b2bbd6b
INFO     This license registration information:

License Type    : pyarmor-group
License No.     : pyarmor-vax-006070
License To      : HEMSCap
License Product : dashboard

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation
  1. Obfuscation

deepdev@deepdev-H270-Gaming-3:~$ touch foo.py
deepdev@deepdev-H270-Gaming-3:~$ pyarmor -d gen --enable-bcc foo.py
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
DEBUG    command options: {'no_runtime': False, 'enable_bcc': True, 'inputs': ['foo.py']}
DEBUG    install plugin: CodesignPlugin
DEBUG    install plugin: DarwinUniversalPlugin
INFO     search inputs ...
INFO     find script foo.py
INFO     find 1 top resources
INFO     start to generate runtime files
INFO     target platforms {'linux.x86_64'}
INFO     write dist/pyarmor_runtime_006070/pyarmor_runtime.so
INFO     patch runtime file
DEBUG    patching runtime data at 747712
DEBUG    patch runtime file OK
DEBUG    call post runtime plugin <class 'pyarmor.cli.plugin.CodesignPlugin'>
INFO     generate runtime files OK
INFO     start to obfuscate scripts
INFO     process resource "foo"
INFO     obfuscating file foo.py
DEBUG    process inline marker
DEBUG    parse script
INFO     start to generate bcc code ...
INFO     no bcc code generated
DEBUG    process co
DEBUG    patch co
INFO     write dist/foo.py
INFO     obfuscate scripts OK
DEBUG    call post build plugin <class 'pyarmor.cli.plugin.DarwinUniversalPlugin'>
  1. I think the error appears when the number of files for obfuscation reaches a
  2. certain threshold
jondy commented 1 year ago

If using same pyarmor, and same logon user, it should no problem.

Could you obfuscate your real scripts by first pyarmor -d reg pyarmor-group-device-xxxx.10.zip then pyarmor -d gen ...?

If it still failed, please paste all the logs of pyarmor -d reg..., and partly logs of pyarmor -d gen... (about the first 20 lines and last 20 lines)

devXpro commented 1 year ago

For two days now, I've been struggling with the activation of a group license. In my case, print(pytransform3.get_hd_info(20)) returns a new value every time, even without rebooting. If you run print(pytransform3.get_hd_info(20)) three times in a row, it will output different strings. Why use such complex mechanisms that are unreliable and require so many manipulations?

Apart from pyarmor, I also have a license for sourceguardian (a similar tool for protecting PHP code). The guys used a mechanism much simpler than this one. And it works reliably and flawlessly. Everything here is as simple as possible.

sourceguardian:
  build:
    context: .
    dockerfile: Dockerfile-sourceguardian
  volumes:
    - /var/run/docker.sock:/var/run/sourceguardian-docker.sock
    - ./src:/src

The external docker.sock is mounted inside the container, and as far as I understand, it generates a machine ID from the external Docker machine.

alikaz3mi commented 1 year ago

@jondy Does pyarmor have file count limitation? Because it works everywhere, except one project which contains 300 files.

First 20 lines:

deepdev@deepdev-H270-Gaming-3:~/PycharmProjects/dashboard_project/dashboard_setup$ pyarmor -d gen -O dapi -r dashboard_api/
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
DEBUG    command options: {'recursive': True, 'output': 'dapi', 'no_runtime': False, 'inputs': ['dashboard_api']}
DEBUG    install plugin: CodesignPlugin
DEBUG    install plugin: DarwinUniversalPlugin
INFO     search inputs ...
INFO     find package at dashboard_api
INFO     find 1 top resources
INFO     start to generate runtime files
INFO     target platforms {'linux.x86_64'}
INFO     write dapi/pyarmor_runtime_006070/pyarmor_runtime.so
INFO     patch runtime file
DEBUG    patching runtime data at 747712
DEBUG    patch runtime file OK
DEBUG    call post runtime plugin <class 'pyarmor.cli.plugin.CodesignPlugin'>
INFO     generate runtime files OK
INFO     start to obfuscate scripts
INFO     process resource "dashboard_api"
INFO     obfuscating file config.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/config.py
INFO     obfuscating file manage.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/manage.py
INFO     obfuscating file main.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/main.py
INFO     obfuscating file config-3.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/config-3.py
INFO     obfuscating file test_configuration.py

Last 20 lines:

INFO     write dapi/dashboard_api/data_matrix_app/domain/entities/package.py
INFO     obfuscating file collected_information.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/data_matrix_app/domain/entities/collected_information.py
INFO     obfuscating file shape.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/data_matrix_app/domain/entities/shape.py
INFO     obfuscating file box.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/data_matrix_app/domain/entities/box.py
INFO     obfuscating file source.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/data_matrix_app/domain/entities/source.py
INFO     obfuscating file __init__.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/api/__init__.py
INFO     obfuscating file apps.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/api/apps.py
INFO     obfuscating file log.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
DEBUG    unknown error, please check pyarmor.error.log
ERROR    invalid license token, try to run `pyarmor reg` to register license again

pyarmor error log:

312145 MainProcess 2023-09-10 09:30:34,724
Traceback (most recent call last):
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__main__.py", line 702, in main
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__main__.py", line 690, in main_entry
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__main__.py", line 229, in cmd_gen
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/generate.py", line 156, in process
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/generate.py", line 119, in _obfuscate_scripts
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/core/__init__.py", line 103, in generate_obfuscated_script
  File "<maker>", line 665, in generate_obfuscated_script
  File "/home/deepdev/.local/lib/python3.10/site-packages/pyarmor/cli/__init__.py", line 16, in process
  File "<maker>", line 504, in process
  File "<maker>", line 510, in coserialize
  File "<maker>", line 555, in _build_ast_body
RuntimeError: invalid license token, try to run `pyarmor reg` to register license again

pyarmor -d reg ...:

deepdev@deepdev-H270-Gaming-3:~$ pyarmor -d reg pyarmor-device-regfile-6070.10.zip 
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
INFO     register "pyarmor-device-regfile-6070.10.zip"
DEBUG    extracting license.lic
DEBUG    extracting .pyarmor_capsule.zip
DEBUG    group license for machines: ['tokens/kcf899c8d241d00b8649f38df8b2bbd6b']
DEBUG    got machine id: kcf899c8d241d00b8649f38df8b2bbd6b
DEBUG    extracting tokens/kcf899c8d241d00b8649f38df8b2bbd6b
INFO     This license registration information:

License Type    : pyarmor-group
License No.     : pyarmor-vax-006070
License To      : HEMSCap
License Product : dashboard

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation
jondy commented 1 year ago

@alikaz3mi

Pyarmor has no file count limitation, it's very strange.

Please help to do a few test?

  1. Does /home/deepdev/.pyarmor/.license.token changed?
  1. If everytime it's log.py failed, try to obfuscate this file only
alikaz3mi commented 1 year ago

The last time it changed was when I ran pyarmor reg. 

deepdev@deepdev-H270-Gaming-3:~/.pyarmor$ ls -la
total 36
drwxrwxr-x  4 deepdev deepdev 4096 aug 21 09:42 .
drwxr-x--- 67 deepdev deepdev 4096 sep 10 12:18 ..
drwxrwxr-x  2 deepdev deepdev 4096 aug  8 11:29 docker
drwxrwxr-x  3 deepdev deepdev 4096 sep  3 08:18 group
-rw-rw-r--  1 deepdev deepdev  197 sep 10 09:30 license.lic
-rw-rw-r--  1 deepdev deepdev  509 sep 10 09:30 .license.token
-rw-rw-r--  1 deepdev deepdev 2835 sep 10 09:30 .pyarmor_capsule.zip
-rw-rw-r--  1 deepdev deepdev  943 sep 10 09:30 pyarmor.debug.log
-rw-rw-r--  1 deepdev deepdev    5 aug 21 09:42 rft_exclude_table

After that, I have ran pyarmor gen for multiple times and it hasn't changed since then.

Doing The steps as requested:

1.

deepdev@deepdev-H270-Gaming-3:~$ pyarmor -d reg pyarmor-device-regfile-6070.10.zip 
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
INFO     register "pyarmor-device-regfile-6070.10.zip"
DEBUG    extracting license.lic
DEBUG    extracting .pyarmor_capsule.zip
DEBUG    group license for machines: ['tokens/kcf899c8d241d00b8649f38df8b2bbd6b']
DEBUG    got machine id: kcf899c8d241d00b8649f38df8b2bbd6b
DEBUG    extracting tokens/kcf899c8d241d00b8649f38df8b2bbd6b
INFO     This license registration information:

License Type    : pyarmor-group
License No.     : pyarmor-vax-006070
License To      : HEMSCap
License Product : dashboard

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation

2.

I was able to obfuscat the log.py with ease

deepdev@deepdev-H270-Gaming-3:~/PycharmProjects/dashboard_project/dashboard_setup$ pyarmor -d gen -O dapi -r dashboard_api/core/models/log.py 
INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64
DEBUG    native platform linux.x86_64
DEBUG    home path: /home/deepdev/.pyarmor
DEBUG    command options: {'recursive': True, 'output': 'dapi', 'no_runtime': False, 'inputs': ['dashboard_api/core/models/log.py']}
DEBUG    install plugin: CodesignPlugin
DEBUG    install plugin: DarwinUniversalPlugin
INFO     search inputs ...
INFO     find script dashboard_api/core/models/log.py
INFO     find 1 top resources
INFO     start to generate runtime files
INFO     target platforms {'linux.x86_64'}
INFO     write dapi/pyarmor_runtime_006070/pyarmor_runtime.so
INFO     patch runtime file
DEBUG    patching runtime data at 747712
DEBUG    patch runtime file OK
DEBUG    call post runtime plugin <class 'pyarmor.cli.plugin.CodesignPlugin'>
INFO     generate runtime files OK
INFO     start to obfuscate scripts
INFO     process resource "log"
INFO     obfuscating file log.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
DEBUG    special co "RERAISE" at <frozen log>:63:save_log_to_db
INFO     write dapi/log.py
INFO     obfuscate scripts OK
DEBUG    call post build plugin <class 'pyarmor.cli.plugin.DarwinUniversalPlugin'>

Although, the error indicates that I have to re-register, but pyarmor -v indicates that my license is still valid. Also, I am able to obfuscate any other file

jondy commented 1 year ago

@alikaz3mi

Got it. I'll do some tests to check it.

jondy commented 1 year ago

@alikaz3mi

I did some tests, but can't reproduce it.

Do you use other options except -r? And do you use any pyarmor cfg to change default options?

alikaz3mi commented 1 year ago

No, I haven't used any other cfg:

INFO     Python 3.10.12
INFO     Pyarmor 8.3.5 (group), 006070, dashboard
INFO     Platform linux.x86_64

------------------------------------------------------------
Section: pyarmor

Current options
  major = 8
  minor = 3
  patch = 5
  cli.core = 4.3.2
  timeout = 6
  regurl = https://api.dashingsoft.com/pr...
  buyurl = https://order.mycommerce.com/p...

Global options

Local options

------------------------------------------------------------
Section: logging

Current options
  debug_logfile = .pyarmor/pyarmor.debug.log
  trace_logfile = .pyarmor/pyarmor.trace.log

Global options

Local options

------------------------------------------------------------
Section: finder

Current options
  recursive = 0
  excludes = */__pycache__
  pyexts = .py .pyw
  data_files = 0
  findall = 0

Global options

Local options

------------------------------------------------------------
Section: builder

Current options
  encoding = utf-8
  enable_trace = 0
  enable_themida = 0
  import_prefix = 0
  bootstrap_file = __file__
  exclude_co_names = <lambda> <listcomp> <setcomp> ...
  exclude_restrict_modules = __init__
  outer_keyname = pyarmor.rkey
  inline_plugin_marker = pyarmor
  plugins = CodesignPlugin DarwinUniversal...
  jit_iv_threshold = 100
  rft_enables = builtin import function class ...
  rft_auto_exclude = 1
  rft_auto_export = 1
  rft_dev_mode = 0
  group_device_flag = 20
  optimize = -1
  type_comments = false
  trace_rft = 0
  enable_jit = 0
  enable_bcc = 0
  enable_rft = 0
  assert_call = 0
  assert_import = 0
  mix_str = 0
  mix_coname = 0
  mix_localnames = 1
  mix_argnames = 0
  obf_module = 1
  obf_code = 1
  wrap_mode = 1
  restrict_module = 1
  import_check_license = 0
  clear_module_co = 1
  clear_frame_locals = 0
  rft_mix_import_name = 0

Global options

Local options

------------------------------------------------------------
Section: runtime

Current options
  universal = 0
  package_name_format = pyarmor_runtime_{suffix}
  simple_extension_name = 1
  outer = 0
  on_error = 0
  nts = pool.ntp.org
  nts_timeout = 3
  timer = 0
  messages = messages.cfg:utf-8

Global options

Local options

------------------------------------------------------------
Section: pack

Current options
  strip = 0
  no_matched_pyc = error

Global options

Local options

------------------------------------------------------------
Section: bcc

Current options
  unsupported_functions = exec eval super locals __asser...
  unsupported_nodes = AsyncFunctionDef AsyncFor Asyn...
  call_function_ex = 0
  trace_lineno = 0
  keep_nest_name = 0
  ignore_lambda = 0
  enable_pure_function = 1
  enable_comprehension = 1

Global options

Local options

------------------------------------------------------------
Section: mix.str

Current options
  threshold = 8

Global options

Local options

------------------------------------------------------------
Section: assert.call

Current options
  auto_mode = and

Global options

Local options

------------------------------------------------------------
Section: assert.import

Current options
  auto_mode = and

Global options

Local options
jondy commented 1 year ago

@alikaz3mi

After I check the source, how about remove the cached tokens and request one again:

rm -rf rm .pyarmor/group/tokens
pyarmor reg -g 10 pyarmor-regfile-6xxx.zip
pyarmor reg pyarmor-group-device-10.zip
pyarmor gen -r /path/to/src
alikaz3mi commented 1 year ago

I have done all of the steps. But issue still remained. Nothing changed. 

DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/data_matrix_app/domain/entities/source.py
INFO     obfuscating file __init__.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/api/__init__.py
INFO     obfuscating file apps.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write dapi/dashboard_api/api/apps.py
INFO     obfuscating file log.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
DEBUG    unknown error, please check pyarmor.error.log
ERROR    invalid license token, try to run `pyarmor reg` to register license again
jondy commented 1 year ago

@alikaz3mi

Could you provide a mini project to reproduce this issue, and send it with reproduce steps to pyarmor@163.com?

jondy commented 1 year ago

@alikaz3mi

And try a clean uninstallation, make sure there is no your own data in ~/.pyarmor, then

mv ~/.pyarmor backup.pyarmor
mv .pyarmor backup.local.pyarmor

pip uninstall pyarmor
pip uninstall pyarmor.cli.core
rm /path/to/package/pyarmor/src

then

pip install pyarmor
pyarmor reg -g 10
pyarmor reg -g 10 xxxx.zip
pyarmor reg pyarmor-group-device-10.zip
pyarmor gen -r /path/to/src
jondy commented 1 year ago

@alikaz3mi

Finally, i reproduce it by obfuscating the whole Python system library.

I'll check it and fix it in next release.

jondy commented 1 year ago

@alikaz3mi

Sorry, I made a mistake, it's still not reproduced.

The error is invalid token, not invalid license token

INFO     obfuscating file py2_test_grammar.py
ERROR    invalid token (<frozen python3.lib2to3.tests.data.py2_test_grammar>, line 31)