foxhunt72 / fhs-xmltv-tools

xmltv tools (downloading, listing channels, removing channels, merging multiple iptv files, listing programs, searching programs) works on linux, windows, macos, raspberry pi, odroid etc.
7 stars 0 forks source link

encoding UTF-8 #3

Open skom27 opened 1 year ago

skom27 commented 1 year ago

Thank you for a great tool :) Unfortunately, I have a problem with merging two xml files. Unfortunately, the files contain characters encoded in UTF-8, where the script stops with an error

e.g. line: Украина 24 HD or the pound sign in the description: "... for her £19,000 budget...." or "4 O'Clock Club" or "Sealladh ùr bho Alba". How can I solve this problem? (PS. I'm unfamiliar with python :( )

foxhunt72 commented 1 year ago

send me two files that you want to merge, and i will see if i can fix it

with regards

richard

skom27 commented 1 year ago

Thank you for your willingness to help :) I just don't know how to send xml files?

skom27 commented 1 year ago

Mayby this way: http://skom.pl/UK.xml http://skom.pl/pl.xml

foxhunt72 commented 1 year ago

Hello,

I tried it with the two files:

fhs-xmltv-tools join-xml-files --xmltv-file pl.xml --xmltv-file-add UX.xml --xmltv-out out.xml fhs-xmltv-tools: 0.1.14

I worked without issue, can you tell me how to what you did to get the error message.

ls -l total 134504 -rw-r--r-- 1 richard richard 69132020 Mar 12 22:05 out.xml -rw-r--r-- 1 richard richard 64870147 Mar 12 21:54 pl.xml -rw-r--r-- 1 richard richard 3726430 Mar 12 21:55 UX.xml

With regards,

Richard

On Thu, Mar 9, 2023 at 10:40 PM skom27 @.***> wrote:

Mayby this way: http://skom.pl/UK.xml http://skom.pl/pl.xml

— Reply to this email directly, view it on GitHub https://github.com/foxhunt72/fhs-xmltv-tools/issues/3#issuecomment-1462861311, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABS4VOQPHBTDU3PVJHKAAJTW3JE3PANCNFSM6AAAAAAVVD5XPU . You are receiving this because you commented.Message ID: @.***>

-- With regards,

Richard

skom27 commented 1 year ago
c:\Download\epg_grab>fhs-xmltv-tools join-xml-files --xmltv-file pl.xml --xmltv-file-add UK.xml --xmltv-out out.xml
fhs-xmltv-tools: 0.1.14
┌─────────────────────────────── Traceback (most recent call last) ────────────────────────────────┐
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\site-packages\fhs_xmltv_tools\cli. │
│ py:251 in join_xml_files                                                                         │
│                                                                                                  │
│   248 │   │   data = join_programs(data, data02)                                                 │
│   249 │                                                                                          │
│   250 │   with console.status("Saving...", spinner="dots"):                                      │
│ > 251 │   │   xmltv_save(xmltv_out, data)                                                        │
│   252                                                                                            │
│   253                                                                                            │
│   254 @main.command()                                                                            │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\site-packages\fhs_xmltv_tools\xmlt │
│ v_load_save.py:32 in xmltv_save                                                                  │
│                                                                                                  │
│   29 │   │   xmltv_data: data object with xmltv data.                                            │
│   30 │   """                                                                                     │
│   31 │   xmltv_out = pathlib.Path(xmltv_file)                                                    │
│ > 32 │   xmltv_helpers.write_file_from_xml(xmltv_out, xmltv_data)                                │
│   33                                                                                             │
│   34                                                                                             │
│   35 def xmltv_empty(source_data_url="", source_info_name="", source_info_url=""):               │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\site-packages\xmltv\xmltv_helpers. │
│ py:70 in write_file_from_xml                                                                     │
│                                                                                                  │
│   67 │   │   no_namespace_schema_location=None))                                                 │
│   68 │                                                                                           │
│   69 │   with xml_file_path.open("w") as data:                                                   │
│ > 70 │   │   serializer.write(data, serialize_clazz)                                             │
│   71                                                                                             │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\site-packages\xsdata\formats\datac │
│ lass\serializers\xml.py:75 in write                                                              │
│                                                                                                  │
│    72 │   │   │   output=out,                                                                    │
│    73 │   │   │   ns_map=namespaces.clean_prefixes(ns_map) if ns_map else {},                    │
│    74 │   │   )                                                                                  │
│ >  75 │   │   handler.write(events)                                                              │
│    76 │                                                                                          │
│    77 │   def write_object(self, obj: Any):                                                      │
│    78 │   │   """Produce an events stream from a dataclass or a derived element."""              │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\site-packages\xsdata\formats\datac │
│ lass\serializers\mixins.py:117 in write                                                          │
│                                                                                                  │
│   114 │   │   │   elif event == XmlWriterEvent.ATTR:                                             │
│   115 │   │   │   │   self.add_attribute(*args)                                                  │
│   116 │   │   │   elif event == XmlWriterEvent.DATA:                                             │
│ > 117 │   │   │   │   self.set_data(*args)                                                       │
│   118 │   │   │   else:                                                                          │
│   119 │   │   │   │   raise XmlWriterError(f"Unhandled event: `{event}`")                        │
│   120                                                                                            │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\site-packages\xsdata\formats\datac │
│ lass\serializers\mixins.py:201 in set_data                                                       │
│                                                                                                  │
│   198 │   │                                                                                      │
│   199 │   │   if value:                                                                          │
│   200 │   │   │   if not self.in_tail:                                                           │
│ > 201 │   │   │   │   self.handler.characters(value)                                             │
│   202 │   │   │   else:                                                                          │
│   203 │   │   │   │   self.tail = value                                                          │
│   204                                                                                            │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\xml\sax\saxutils.py:213 in         │
│ characters                                                                                       │
│                                                                                                  │
│   210 │   │   │   self._finish_pending_start_element()                                           │
│   211 │   │   │   if not isinstance(content, str):                                               │
│   212 │   │   │   │   content = str(content, self._encoding)                                     │
│ > 213 │   │   │   self._write(escape(content))                                                   │
│   214 │                                                                                          │
│   215 │   def ignorableWhitespace(self, content):                                                │
│   216 │   │   if content:                                                                        │
│                                                                                                  │
│ C:\Users\Krzysiek\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1250.py:19 in encode   │
│                                                                                                  │
│    16                                                                                            │
│    17 class IncrementalEncoder(codecs.IncrementalEncoder):                                       │
│    18 │   def encode(self, input, final=False):                                                  │
│ >  19 │   │   return codecs.charmap_encode(input,self.errors,encoding_table)[0]                  │
│    20                                                                                            │
│    21 class IncrementalDecoder(codecs.IncrementalDecoder):                                       │
│    22 │   def decode(self, input, final=False):                                                  │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-6: character maps to <undefined>

With regards Chris