al-caughey / YAMon-v4

Official repository for YAMon v4
https://usage-monitoring.com
62 stars 21 forks source link

ISP Import Format From Excel/CSV/TSV #3

Open OzTechGeek opened 4 years ago

OzTechGeek commented 4 years ago

What format is the ISP Monthly Import function looking for? I'm trying to import my usage from an excel spreadsheet I created, but get the following error when trying to import:

There was a problem computing the month (00) or year (undefined) values ?!?

I've tried various date formats in excel and pasted the resuls in the import box but it continues to fail:

Import Sample:

Date        Download    Upload
2020-02-08  14.804      0.000
2020-02-10  4.417       0.000
2020-02-11  4.819       0.000
2020-02-12  4.379       0.000
2020-02-13  4.871       0.000
al-caughey commented 4 years ago

What format is the ISP Monthly Import function looking for? I'm trying to import my usage from an excel spreadsheet I created, but get the following error when trying to import: Import Sample:

Date      Download    Upload
2020-02-08    14.804      0.000
2020-02-10    4.417       0.000
2020-02-11    4.819       0.000
2020-02-12    4.379       0.000
2020-02-13    4.871       0.000

The ISP related code is mostly in util4.0.js

function addISPList(){} [lines 3046-3060] --> adds the list of ISPs to the Settings tab

And, more importantly,

$('#process-isp').click(function(){...}) [line 1306-1429] --> interprets the content pasted into the textarea... usually people copy the text directly from the page at their ISP and then paste it directly into the textarea so the bulk of the code above deals with differences in formatting and units. Basically there are several regular expressions to chunk the data into date, down & up and then corrections for date format & then units (MB vs GB, etc).

Given that you've already done the chunking, it'll be really simple to add yours... what's your ISPs name? Are the daily totals in GB?

OzTechGeek commented 4 years ago

My ISP is AT&T, my issue is that this is not a "normal" ISP internet connection but rather an AT&T internet connection using a mobile hotspot, just like regular data on a phone, so the usage reports from myAT&T account is based on mobile data usage so it's broken down on a connection by connection basis, so for each day I have multiple entries in KB, so that is why I export the data from myAT&T to a CSV and then used an Excel Power Query to group all dates together so I could have an entry for each day then totaled the daily usage in KB and divided it by 1024 to get my MB per day, also AT&T does not provide download and upload, just a total usage number, that is why I created my own import data with "Upload" at 0

This is what the actual RAW data from the MyAT&T CSV Export looks like:

Item,Day,Date,Time,To/From,Type,Msg/KB,Rate Code,Rate Pd,Feature,In/Out,Total Charge 
1,Saturday,02/08/2020,02:52AM,Data Transfer,,"4,482 KB",SDGU,AT,GPRR,,0.00
2,Saturday,02/08/2020,8:57PM,Data Transfer,,"10,677 KB",SDGU,AT,GPRR,,0.00
3,Monday,02/10/2020,02:33AM,Data Transfer,,"4,523 KB",SDGU,AT,GPRR,,0.00
4,Tuesday,02/11/2020,02:53AM,Data Transfer,,"4,935 KB",SDGU,AT,GPRR,,0.00
5,Wednesday,02/12/2020,02:49AM,Data Transfer,,"4,484 KB",SDGU,AT,GPRR,,0.00
6,Thursday,02/13/2020,02:31AM,Data Transfer,,"4,988 KB",SDGU,AT,GPRR,,0.00

Hope this helps. Thanks again for your time

al-caughey commented 4 years ago

So you can get the 'raw' CSV export directly from your MyAT&T page? If yes, rather than pre-processing in Excel, I/you can write a little function that a) creates an array of the lines (split('\n') b) splits each of the lines into fields (split(',')) date=line[2] & usge=line[6].replace(',','').replace(' KB','') c) totals the lines by date total[date]+=usage*1

It's simple when you say it fast... if you want, send me the raw file

OzTechGeek commented 4 years ago

@al-caughey Thanks for your help, I have attached a CSV extract (renamed to txt) from a MyAT&T CSV export for the internet usage for last month. Thanks

CSV Export: Usage_2020-03.txt