UnicodeEncodeError: 'charmap' codec can't encode characters in position 1944-1945: character maps to <undefined>

[mohitmishra786]

Failed to post to Medium Returned error message below

"UnicodeEncodeError: 'charmap' codec can't encode characters in position 1944-1945: character maps to

Traceback (most recent call last): File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_bundler.py\", line 37, in upload data = publish(**kwargs) File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_publish_to_medium.py\", line 300, in publish p.main() File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_publish_to_medium.py\", line 218, in main self.md, self.image_data_dict = self.create_markdown() File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_publish_to_medium.py\", line 121, in create_markdown no_ex_pp.preprocess(self.nb, self.resources) File \"C:\Users\mohit\anaconda3\lib\site-packages\nbconvert\preprocessors\base.py\", line 69, in preprocess nb.cells[index], resources = self.preprocess_cell(cell, resources, index) File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_preprocesors.py\", line 148, in preprocess_cell output['data'] = {'image/png': converter(html)} File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_screenshot.py\", line 169, in run img = self.take_screenshot() File \"C:\Users\mohit\anaconda3\lib\site-packages\jupyter_to_medium\_screenshot.py\", line 97, in take_screenshot f.write(self.html) File \"C:\Users\mohit\anaconda3\lib\encodings\cp1252.py\", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode characters in position 1944-1945: character maps to

[mohitmishra786]

@tdpetrou @mjam03 check this issue once please

[mjam03]

Hi,

It looks like there is an issue with your notebook when it comes to converting a dataframe table to an image. Without a notebook to check it against there's not much I can do. Can you attach a notebook please so I can check what exactly is failing?

[mohitmishra786]

Data Exploration

This was the notebook, I even tried reading CSV file without r before file path but stills the issue was same. @mjam03 @tdpetrou

[mjam03]

Okay let me have a look and see if I can replicate the issue - thanks for the notebook

[mohitmishra786]

@mjam03 did you got something done

[mjam03]

I'm working on it right now

[mjam03]

So I've done the following and have managed to upload the notebook to Medium with no problems. Can you please try and do the following and see if the problem is solved?

I am running this on my mac btw

# create a new test environ and go into it
mkdir ~/dev/jtm_test
cd ~/dev/jtm_test
# curl the notebook into it
curl -LJO https://raw.githubusercontent.com/mohitmishra786/Machine-Learning-Models/main/Natural-Language-Processing/Hate%20Speech%20Detection/Data%20Exploration.ipynb
# rename the notebook to remove special chars
mv Data%20Exploration.ipynb Data\ Exploration.ipynb
# create a new conda env to test
conda env create --file environment.yml
# activate the environment called jtm_test
conda activate jtm_test
# ensure extension is added
jupyter bundlerextension enable --py jupyter_to_medium._bundler --sys-prefix
# start jupyter notebook session and try to upload
jupyter notebook

From here then open the notebook and run Deploy As -> Medium and it should work with no problems.

The environment.yml file that the environment is created on is here:

name: jtm_test
channels:
  - defaults
dependencies:
  - python>=3.6
  - beautifulsoup4
  - jupyter
  - matplotlib>=3.1
  - nbconvert
  - numpy
  - pip
  - requests
  - pip:
    - jupyter_contrib_nbextensions
    - jupyter_to_medium

[mohitmishra786]

@mjam03 after copying all these steps even now I am getting the same error

Failed to post to Medium
Returned error message below

"UnicodeEncodeError: 'charmap' codec can't encode characters in position 1944-1945: character maps to

Traceback (most recent call last):
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_bundler.py\", line 37, in upload
data = publish(**kwargs)
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_publish_to_medium.py\", line 300, in publish
p.main()
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_publish_to_medium.py\", line 218, in main
self.md, self.image_data_dict = self.create_markdown()
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_publish_to_medium.py\", line 121, in create_markdown
no_ex_pp.preprocess(self.nb, self.resources)
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\nbconvert\\preprocessors\\base.py\", line 69, in preprocess
nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_preprocesors.py\", line 148, in preprocess_cell
output['data'] = {'image/png': converter(html)}
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_screenshot.py\", line 169, in run
img = self.take_screenshot()
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\site-packages\\jupyter_to_medium\\_screenshot.py\", line 97, in take_screenshot
f.write(self.html)
File \"C:\\Users\\mohit\\anaconda3\\envs\\jtm_test\\lib\\encodings\\cp1252.py\", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 1944-1945: character maps to
"```

[mohitmishra786]

@tdpetrou @mjam03 can you check over this please

[mjam03]

I really can't do much if I can't replicate the issue. I've tried:

• uploading from the jupyter notebook interface (works)
• uploading from the command line (works)

And the package passes all of the tests for python 3.7-3.9 across all operating systems. It also seems like you have Chrome available for the screenshots otherwise it would throw an error saying as such. All I can offer is that you:

• try removing certain tables and see which table is causing the issue
• try uploading just the first half and last half etc and continue binary search style to see what's causing the issue

Maybe then I can try and figure out what is going wrong for a specific table but again this is a challenge unless I can replicate. I'm going to close the issue for now as there is little I can do to debug and help.