search

birkenfeld / serde-pickle

Rust (de)serialization for the Python pickle format.
Apache License 2.0
188 stars 28 forks source link

Option to extract fields from class #5

Closed euclio closed 5 years ago

euclio commented 5 years ago

I'd like to deserialize a pickle that contains a class. I see that this is unsupported by the library, I'm assuming because it's difficult to map Python classes to Rust structs: https://github.com/birkenfeld/serde-pickle/blob/639592fda875a29ec26eb4970179396e9c11f0a7/src/consts.rs#L68-L78

Particularly, my pickle fails to deserialize on the i opcode. However, it would be useful to me to extract just the fields of the class (without methods, etc.) into a struct/Value. Would it be feasible to provide such an option?

birkenfeld commented 5 years ago

I think it would be possible in many cases, but I'm a bit rusty on the details. AFAIR some classes can provide a custom reconstructor with custom arguments, this would need to remain unsupported. And as you can see from the number of op codes, there are a lot of compatibility layers ;)

If you are able to attach a sample pickle, I can make sure a change I make works at least for your use case.

euclio commented 5 years ago

I'm trying to deserialize deluge's state pickle:

(ideluge.core.torrentmanager
TorrentManagerState
p1
(dp2
S'torrents'
p3
(lp4
(ideluge.core.torrentmanager
TorrentState
p5
(dp6
S'max_download_speed'
p7
F-1
sS'move_completed_path'
p8
S'/home/euclio/Downloads'
p9
sS'paused'
p10
I00
sS'max_upload_slots'
p11
I0
sS'prioritize_first_last'
p12
I01
sS'max_connections'
p13
I-1
sS'compact'
p14
I00
sS'queue'
p15
I-1
sS'file_priorities'
p16
(lp17
I1
asS'filename'
p18
S'archlinux-2017.10.01-x86_64.iso.torrent'
p19
sS'max_upload_speed'
p20
F0
sS'save_path'
p21
S'/home/euclio/down/torrents'
p22
sS'time_added'
p23
F1507503943.682688
sS'total_uploaded'
p24
I0
sS'torrent_id'
p25
S'9228628504cc40efa57bf38e85c9e3bd2c572b5b'
p26
sS'auto_managed'
p27
I01
sS'stop_at_ratio'
p28
I00
sS'move_completed'
p29
I00
sS'trackers'
p30
(lp31
(dp32
S'send_stats'
p33
I01
sS'fails'
p34
I0
sS'verified'
p35
I00
sS'scrape_incomplete'
p36
I-1
sS'min_announce'
p37
NsS'source'
p38
I1
sS'url'
p39
S'http://tracker.archlinux.org:6969/announce'
p40
sS'last_error'
p41
(dp42
S'category'
p43
S'system'
p44
sS'value'
p45
I0
ssS'fail_limit'
p46
I0
sS'next_announce'
p47
NsS'complete_sent'
p48
I00
sS'scrape_downloaded'
p49
I-1
sS'trackerid'
p50
S''
sS'start_sent'
p51
I00
sS'tier'
p52
I0
sS'scrape_complete'
p53
I-1
sS'message'
p54
S''
sS'updating'
p55
I00
sasS'magnet'
p56
NsS'remove_at_ratio'
p57
I00
sS'stop_ratio'
p58
F2
sS'is_finished'
p59
I01
s

Thank you for looking into this!

birkenfeld commented 5 years ago

Ok, I pushed version 0.5 which should take care of that structure. Let me know!

euclio commented 5 years ago

Works great. Thanks for your work!