Open mutr opened 3 years ago
ghost
commented
3 years ago First to lines, feel free to check for errors
(This file must be converted with BinHex 4.0)
:$QaKD@jIEfeKDf8ZCfPQ*%G*4QBi3NPO!3!!!!Rq!!!#L4aa4dP!I$PKPJ$L!)!
!!2rrr~!!!#~!!!!!PJ$L!!!#ri52U4(VlaS+%YU,XpkmHdOpiNL@*Pl*kFQfEKYDecoding that header will result in a lain_omake.gif file. rest of the code would be image data
h5vx
commented
3 years ago Googling first line leads me to this site http://www.noisywires.net/layer:09/layer:09.html
ghost
commented
3 years ago Looks like he abandoned it way before we picked it up.
Also file 2 lines:
(This file must be converted with BinHex 4.0)
:$'PZEQpMC@jd,QT`C`"+8%9(1%**638!!!!,1J!!!!!C%2rBrq!!%%T'58B!!3)
"!5`",!!!rqd##&"SEh4[FfK[F#!c,M!!1%**632T!!!!!!"i!!B!!!")!%J!!!!
!
Agapurnis
commented
3 years ago It may be useful to identify the font so that it can be overlaid. Just doing the first line by hand, I'm already getting some differences when compared to the first line on the website, which isn't a good sign. However, that might just be me.
ghost
commented
3 years ago For clarity, im posting all available characters in BinHex 4.0
!"#$%&'()*+,-012345689@ABCDEFGHIJKLMNPQRSTUVXYZ[`abcdefhijklmpqr
mutr
commented
3 years ago @Agapurnis Post your progress in the comments.
The font is one of Courier (typeface) variant. I will try to make a dataset for neural network from this font. I am interested in making a fully automated solution. I hope to find time for this on Saturday.
rikkaonline
commented
3 years ago Guys you're not going to find anything useful. It's the psd file of the image converted to text and overlaid on the picture. The table of contents of rebuild explains it.
mutr
commented
3 years ago @rikkaonline The TOK just says that image inside. We already know the file names. It's lain_omake.gif and innocent.jpg.
As you can see it's not even .psd.
What do you mean by useful? Did you hear about "hack value"?
Hack value is the notion used by hackers to express that something is worth doing or is interesting. This is something that hackers often feel intuitively about a problem or solution. An aspect of hack value is performing feats for the sake of showing that they can be done, even if others think it is difficult.
This files is part of the history of Lain fandom. The content of the file is not really matter. If something is encoded it must be decoded. I enjoy the process. For me it's like a 20 years old puzzle. А time capsule.
rikkaonline
commented
3 years ago Here's what the book says:
The psd filename is right there.
You're not going to get anything useful in the sense that there is not a message or "code" hidden there to be cracked. He wanted to throw some random text on the pic to look cool so he ran the psd file through a tool to get that. If you want to run ocr on it to see what you can accomplish then sure have fun, but it's not going to yield data that has intrinsic value, and its certainly not worth the time sink of transcribing it all by hand.
mutr
commented
3 years ago Please look at other pages in TOC. Every layer has .psd not just 09. It's not related to content of the encoded file, it's related to page itself.
You can easily check that you are wrong. Try to decode the fragments we posted here.
First several bytes of the first file: lain_omake.gif\x00GIFf8BIN\x01
As you can see file has GIF signature and extension.
GIF files start with a fixed-length header ("GIF87a" or "GIF89a") giving the version, followed by a fixed-length Logical Screen Descriptor giving the pixel dimensions and other characteristics of the logical screen.
import base64
ref = '!"#$%&\'()*+,-012345689@ABCDEFGHIJKLMNPQRSTUVXYZ[`abcdefhijklmpqr'
mytrans = str.maketrans(ref, 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/')
base64.b64decode(inp.translate(mytrans).encode())
inp=':$QaKD@jIEfeKDf8ZCfPQ*%G*4QBi3NPO!3!!!!Rq'
base64.b64decode(inp.translate(mytrans).encode())Easy peasy.
You're not going to get anything useful in the sense that there is not a message or "code" hidden there to be cracked.
I definitely don't expect to find the "code" that let me become Knights members or hack the pentagon. Ha-ha.
and its certainly not worth the time sink of transcribing it all by hand.
If you do not see value in something, this does not mean that it does not exist. If person A transcribed a file by hand and checksum matches then he or she will be the first in the world who do this. I will accept his PR right away and his name will be in this repository forever. Everyone who will google this code will see that a specific person A was the first.
If someday person B can suggest a better and more elegant solution requiring less hand work I will glad to add his code too. This will not cancel that the person A was first. That's the value. If you can't see this value just don't do it. It's not for you. Do something that has value to you.
rikkaonline
commented
3 years ago Of course every page has a psd file, every page is an image. That's the filename of the image on that page. It's related to the content of the encoded file by the text description that literally says that's what the text is.
You can easily check that you are wrong. Try to decode the fragments we posted here. A psd file could contain the contents of a gif file somewhere in it if that file is imported into it.
mutr
commented
3 years ago You still don't understand. It's not just a file or an artwork. It's a MESSAGE. The message from authors of Lain to those who can find it, open it and understand it. When someone open it that means that he or she was the recipient. It's like a catch message in a bottle. The content is not matter at all because recipient already know what this message is about. This message is about importance of information. If you're not interested just move it to the spam folder.
rikkaonline
commented
3 years ago mutr
commented
3 years ago Oh, man. You are boring :)
mutr
commented
3 years ago I generate a dataset from background and Courier font and train a simple NN on it.
Works perfect on test data but not on real. Need to add variability and switch to CNN.
Real data:
ghost
commented
3 years ago Thats nice also here are the lines from the second pic, without the last 10 lines. some characters could be others, i also included ? mark if the character was unreadable
(This file must be converted with BinHex 4.0)
:$'PZEQpMC@jd,QT`C`"+8%9(1%**638!!!!,1J!!!!!C%2rBrq!!%%T'58B!!3)
"!5`",!!!rqd##&"SEh4[FfK[F#!c,M!!1%**632T!!!!!!"i!!B!!!")!%J!!!!
!
nekto-nekto
commented
3 years ago Maybe you preliminarily make clear the picture from printing/scans artifacts? Example: https://photo.stackexchange.com/questions/23445/what-is-the-best-way-to-remove-texture-from-a-scanned-textured-photo-paper
kostapc
commented
3 years ago I was trying to remove scan (printing) artifacts, picture became better, but text becomes significant less readable.
mutr
commented
3 years ago Partial success. Changed one byte on DHT table. Picture can be opened now, but still looks corrupted.
I changed decoder a bit, now it saves the file before RLE decoding (before_rle.jpg). It seems RLE is not used by the encoder. And it can produce garbage on corrupted parts.
Courier is worst font for OCR ever. Look at this:
Yes that symbols are almost the same. And there is a lot of "l" and "1" mixed in the second half of a file.
h5vx
commented
3 years ago Playing around 1 / l, got little better results. Now produced JPEG can be opened without touching it's bytes
nekto-nekto
commented
3 years ago ghost
commented
3 years ago 3
- TODO: Use https://github.com/google/knusperli for minimize JPEG blocking artefact or maybe try this deblocking filter: https://www.compression.ru/video/deblocking/smartdeblocking_en.html And try waifu2x (or another NN) for enhance resolution.
Both of these methods should be used AFTER we get the final result.
nekto-nekto
commented
3 years ago @lelenium
Both of these methods should be used AFTER we get the final result.
Of course.
mutr
commented
3 years ago Uploaded a second file - raw_files/lain_omake.gif.hqx
BinHex header checksum is valid. It's GIF image 150x226px black and while. Image is still broken.
Also this file contains resource fork.
nick758
commented
2 years ago Hi!
Thank you for your work.
I fixed OCR errors in the file lain_omake.gif.hqx and got correct lain_omake.gif. CRC is OK. I didn't fixed resource fork because I don't know its format.
Fixed hqx is here: lain_omake.gif.zip
mutr
commented
2 years ago This is so awesome! Thank you! Are you going to create pull request?
What kind of object is she holding in her lap? Any ideas?
GermanAizek
commented
2 years ago I fixed OCR errors in the file lain_omake.gif.hqx and got correct lain_omake.gif. CRC is OK. I didn't fixed resource fork because I don't know its format.
@nick758, cool, I did not believe that soon would transfer the binary file from the paper to the file.
nick758
commented
2 years ago @GermanAizek It was semi-manual brute forcing :) I made utility which generates modified files with combinations of suspicious symbols. It receive number of changes and offset where to start. Default suspicious symbols is '"` and l1 sometimes I added ! in the first group. So, I generated about 100-200 files (about 5 changes in a file) at a time then decoded them and selected the better one visually. If result didn't become better I looked for mistakes in other symbols. And repeated this procedure until end.
nick758
commented
2 years ago @mutr
What kind of object is she holding in her lap? Any ideas?
Maybe it's a packet with a gun which is seen in the game in Dc1057: https://laingame.net/index.php?id=55 Also Lain with a gun in Lda230: https://laingame.net/index.php?id=418
nick758
commented
2 years ago innocent.jpg.hqx I've checked symbols which are duplicated in bigger font on the whole picture: https://i.warosu.org/data/lit/img/0052/57/1407621588546.jpg There were 3 mistakes but CRC is still wrong and there are no visual changes on the picture.
Pull request is created.
mutr
commented
2 years ago Thank you!
nick758
commented
2 years ago Hi guys! I made time-lapse of fixing errors in "lain_omake.gif.hqx". I don't know why I did it. But let it be here: https://www.youtube.com/watch?v=eqRgU7CtQPM
At first we need to convert the text on images to plain text. I tried some OCRs but text is too small and fuzzy. Maybe someone can do it using only own eyes and hands. Or even train neural network to recognize it. Sounds like a interesting challenge. #lain_artbook_code_challenge
The second stage is decoding. The file format is BinHex 4.0 It used in classic macs. It's is very easy format that includes Base64-like encoding and RLE compression. Nothing serious. Macutils can open it for example. We already tried to decode the first line and it looks amazing. The only problem is that the code does not contains the error correction mechanism, only checksum. Some symbols are too fuzzy and barely can be seen. There will be errors all over and we have to convert them by hand or by brute force algorithm.