Closed Saisuma004 closed 6 years ago
That totally make sense I was looking only at #6 issue URL. I will be working on that.
From: Torrin Hultgren [mailto:notifications@github.com] Sent: Tuesday, September 19, 2017 1:42 PM To: USEPA/EPA_Environmental_Dataset_Gateway EPA_Environmental_Dataset_Gateway@noreply.github.com Cc: Malothu, Sumalatha Sumalatha.Malothu@Woolpert.com; Author author@noreply.github.com Subject: Re: [USEPA/EPA_Environmental_Dataset_Gateway] Replaced special character to dash(-). #6 (#31)
@torrin47 commented on this pull request.
I'm really wary of doing any explicit character replacement - because that's like playing whack-a-mole. There will always be other characters to replace - like the funky quotes in this ticket: #25https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUSEPA%2FEPA_Environmental_Dataset_Gateway%2Fissues%2F25&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=4MgkB2OfwnWWBd%2BgvdIY2VP6Cxlp7ea5tCuIqhEwZcg%3D&reserved=0 In reading through the full InnoRestServlet.java file, I see that the code actually sends a request to a separate EsriRESTServlet to obtain the raw data. So I compared the response of that servlet with the Innovate servlet: Esri: https://edg.epa.gov/metadata/ESRIRestServlet?id={C6AE0507-D98E-45CB-8B8B-B421731BA330}https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedg.epa.gov%2Fmetadata%2FESRIRestServlet%3Fid%3D%257BC6AE0507-D98E-45CB-8B8B-B421731BA330%257D&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=ho4kOAIBlUXI%2BPnB0OiCdFbrcU8nLIaCc8jpwuNiQsY%3D&reserved=0 Inno: https://edg.epa.gov/metadata/rest/document?id={C6AE0507-D98E-45CB-8B8B-B421731BA330}https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedg.epa.gov%2Fmetadata%2Frest%2Fdocument%3Fid%3D%257BC6AE0507-D98E-45CB-8B8B-B421731BA330%257D&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=4Iq87CSnVaAZZoDYjXo49IzIx%2BWf1OApIIAVYgZtjkY%3D&reserved=0 Note the different quotes in the title, even though both send the same content-type header (Content-Type:text/xml;charset=UTF-8). So there's something about the innovate code that's not accepting the input it gets from the EsriRESTServlet as UTF-8. I see that in line 296 there's a BufferedReader that gets data from an InputStreamReader with no encoding set. This seems like it might be the issue. https://stackoverflow.com/questions/2260325/why-is-java-bufferedreader-not-reading-arabic-and-chinese-characters-correctlyhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F2260325%2Fwhy-is-java-bufferedreader-not-reading-arabic-and-chinese-characters-correctly&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=Mz2yFQ7TTNFHTEpQFIude705PDM3iAUq3%2BHHKtle2d0%3D&reserved=0
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUSEPA%2FEPA_Environmental_Dataset_Gateway%2Fpull%2F31%23pullrequestreview-63761601&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=B8DUJGgyY6gFvoljxm2JjT037gjUWV2aUEpaKuClSxc%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAQVVNkMc5i2LO6Kn0itOKNVrtRA5ZhJxks5skArEgaJpZM4PcoNR&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=KfhVoBAxl8DG%2FerLMX6ZIcoiZ9%2Fj35qYYTdgQrixIGY%3D&reserved=0.
That totally make sense I was looking only at #6 issue URL. I will be working on that.
From: Torrin Hultgren [mailto:notifications@github.com] Sent: Tuesday, September 19, 2017 1:42 PM To: USEPA/EPA_Environmental_Dataset_Gateway EPA_Environmental_Dataset_Gateway@noreply.github.com Cc: Malothu, Sumalatha Sumalatha.Malothu@Woolpert.com; Author author@noreply.github.com Subject: Re: [USEPA/EPA_Environmental_Dataset_Gateway] Replaced special character to dash(-). #6 (#31)
@torrin47 commented on this pull request.
I'm really wary of doing any explicit character replacement - because that's like playing whack-a-mole. There will always be other characters to replace - like the funky quotes in this ticket: #25https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUSEPA%2FEPA_Environmental_Dataset_Gateway%2Fissues%2F25&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=4MgkB2OfwnWWBd%2BgvdIY2VP6Cxlp7ea5tCuIqhEwZcg%3D&reserved=0 In reading through the full InnoRestServlet.java file, I see that the code actually sends a request to a separate EsriRESTServlet to obtain the raw data. So I compared the response of that servlet with the Innovate servlet: Esri: https://edg.epa.gov/metadata/ESRIRestServlet?id={C6AE0507-D98E-45CB-8B8B-B421731BA330}https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedg.epa.gov%2Fmetadata%2FESRIRestServlet%3Fid%3D%257BC6AE0507-D98E-45CB-8B8B-B421731BA330%257D&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=ho4kOAIBlUXI%2BPnB0OiCdFbrcU8nLIaCc8jpwuNiQsY%3D&reserved=0 Inno: https://edg.epa.gov/metadata/rest/document?id={C6AE0507-D98E-45CB-8B8B-B421731BA330}https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedg.epa.gov%2Fmetadata%2Frest%2Fdocument%3Fid%3D%257BC6AE0507-D98E-45CB-8B8B-B421731BA330%257D&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=4Iq87CSnVaAZZoDYjXo49IzIx%2BWf1OApIIAVYgZtjkY%3D&reserved=0 Note the different quotes in the title, even though both send the same content-type header (Content-Type:text/xml;charset=UTF-8). So there's something about the innovate code that's not accepting the input it gets from the EsriRESTServlet as UTF-8. I see that in line 296 there's a BufferedReader that gets data from an InputStreamReader with no encoding set. This seems like it might be the issue. https://stackoverflow.com/questions/2260325/why-is-java-bufferedreader-not-reading-arabic-and-chinese-characters-correctlyhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F2260325%2Fwhy-is-java-bufferedreader-not-reading-arabic-and-chinese-characters-correctly&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=Mz2yFQ7TTNFHTEpQFIude705PDM3iAUq3%2BHHKtle2d0%3D&reserved=0
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUSEPA%2FEPA_Environmental_Dataset_Gateway%2Fpull%2F31%23pullrequestreview-63761601&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=B8DUJGgyY6gFvoljxm2JjT037gjUWV2aUEpaKuClSxc%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAQVVNkMc5i2LO6Kn0itOKNVrtRA5ZhJxks5skArEgaJpZM4PcoNR&data=01%7C01%7Csumalatha.malothu%40woolpert.com%7C0612af8051fb4537438e08d4ff8e112a%7C49c1e384218e47a1a464d59d76daf482%7C0&sdata=KfhVoBAxl8DG%2FerLMX6ZIcoiZ9%2Fj35qYYTdgQrixIGY%3D&reserved=0.