use sys.stdin.buffer to work around type mismatch in codecs.py

[xiota]

This fixes an issue with reading input from stdin.

---

When I run screenplain --format=html to take input from stdin, I get the following error:

Traceback (most recent call last):
  File ".local/bin/screenplain", line 8, in <module>
    sys.exit(cli())
  File ".local/lib/python3.9/site-packages/screenplain/main.py", line 140, in cli
    main(sys.argv[1:])
  File ".local/lib/python3.9/site-packages/screenplain/main.py", line 99, in main
    screenplay = fountain.parse(input)
  File ".local/lib/python3.9/site-packages/screenplain/parsers/fountain.py", line 219, in parse
    content = stream.read()
  File "/usr/lib/python3.9/codecs.py", line 500, in read
    data = self.bytebuffer + newdata
TypeError: can't concat str to bytes

This occurs with the versions installed with pip or from git (bc719b6).

Changing the relevant line in codecs.py fixes the problem.

data = self.bytebuffer + bytes(newdata, 'utf-8')

So does using sys.stdin.buffer in screenplain/main.py.

[vilcans]

Thank you, I hadn't noticed that!

[xiota]

@vilcans I noticed the problem while trying to use screenplain as an auxiliary processor for a text editor plugin. It doesn't error out now, but I'm still having trouble getting it to work with the plugin (written in C). Any thoughts about potential causes for the problem?

screenplain now seems to work as expected when input is piped from a terminal:

  screenplain --format=html < Big-Fish.fountain
  cat Big-Fish.fountain | screenplain --format=html

However, I've noticed when typing directly into stdin, I have to press Ctrl+D twice to close input. It's the same behavior as when I modify codecs.py.

---

Edit: Found this explanation on stackexchange. Would changing screenplain to work around this issue be a merge request that you'd be interested in? (If not, I was thinking about writing a wrapper program in C/C++.)

Edit: Found the problem with the plugin... working now... Thanks for the great program.

[vilcans]

Yeah, it's strange that you have to press Ctrl+D twice. I wonder if it's wrong to force utf-8 the way we do when opening stdin. Perhaps that line should be simply input = sys.stdin.

On Thu, Sep 16, 2021 at 1:34 AM xiota @.***> wrote:

@vilcans https://github.com/vilcans I noticed the problem while trying to use screenplain as an auxiliary processor for a text editor plugin https://github.com/xiota/geany-preview. It doesn't error out now, but I'm still having trouble getting it to work with the plugin (written in C). Any thoughts about potential causes for the problem?

screenplain now seems to work as expected when input is piped from a terminal:

screenplain --format=html < Big-Fish.fountain cat Big-Fish.fountain | screenplain --format=html

However, I've noticed when typing directly into stdin, I have to press Ctrl+D twice to close input. It's the same behavior as when I modify codecs.py.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vilcans/screenplain/pull/70#issuecomment-920461551, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAL4ALHM5BPKFTU643DCX3UCEUP7ANCNFSM5EC2C5YQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[xiota]

I'm guessing the Ctrl+D twice problem is related to codecs package. I just tried input = sys.stdin, and it works perfectly. It looks like strings in Python 3 are already utf-8, so it isn't needed to use a separate library for it unless you need to support Python 2?