search

rendicott / php-ipam-cli

CLI interface to phpIpam
13 stars 4 forks source link

MAC address as an input field #1

Closed power8ce closed 8 years ago

power8ce commented 8 years ago

Would it be possible to add MAC address as one of the input field for import? Thanks!

rendicott commented 8 years ago

You're referring to the built in MAC address field within phpIpam and not a custom attribute right?

power8ce commented 8 years ago

Yes. Would it be possible to include another column for MAC?

Ip,Name,Metadata,Description,ReservedBy,ReservedOn,IsProduction,IsTaken,cdl_isprod,cdl_autodelete,cdl_dhcp,MAC (example)

rendicott commented 8 years ago

@power8ce check out the new branch 'feature/mac-addr-import'. You should be able to just add a 'mac' column to your import file and it will populate phpIpam's 'MAC Address' field now. See README in that branch for an example csv. If it works out let me know and I'll merge to master.

power8ce commented 8 years ago

I'm getting some error. Mind help me debug? I can't read Python... :)

I'm not utilizing the HTTPs auth part. I did disable secure auth by setting "$enable_authentication = false;" in /var/www/html/phpipam/api/v2/index.php file.

# python pysser.py -c runningconfig.ini --addressfile=inventory.csv
Traceback (most recent call last):
  File "pysser.py", line 1057, in <module>
    main(options)
  File "pysser.py", line 892, in main
    (settings, session) = dataload(opts)
  File "pysser.py", line 873, in dataload
    session = authenticate(settings, session)
  File "pysser.py", line 81, in authenticate
    string_token = str(r.json().get('data').get('token'))
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
# cat runningconfig.ini
[global]
# https is required. All API calls expect it.
#base_url: https://phpipam.mycompany.com/phpipam
base_url: http://192.168.7.240/phpipam
# the app_id that you have set up in your phpIpam API applications
#  see: http://phpipam.net/api-documentation/
app_id: CLI
username: Admin
password: L@bTEST1
# the default section in which to interact with the API
default_section: DEFAULT
# cat inventory.csv
192.168.7.200,TESTTEST,USED,awesome box,00:45:EF:1A:6C:AD,0

Thank you for any insight.

rendicott commented 8 years ago

I just tested it with HTTP and it seems to be working fine (I should update the documentation). I think there's a mismatch in how you have your 'Application' set up within phpIpam. In the 'Administration' page for phpIpam there's an 'Applications' section where you have to set up an application endpoint to enable the API. In your above config sample you have your application id set to 'CLI'. So using the below screenshot you should set up an application called 'CLI' with 'App security' set to 'None'.

image

Also, make sure you actually have a section called 'DEFAULT' within phpIpam. In the below screenshot there are two sections, one of them is called 'TESTING' which is circled in red.

image

power8ce commented 8 years ago

Yea... I already do have both API and Section settings set correct. I just double-checked again.

I tried creating a new API App id and new user. Still the same.

Is there some debugging mode I can run? I tried supplying argument "-d ?" (? being the numbers according to your doc) it doesn't produce anything more of an error.

I'll see if I can get another instance up and running see if it'll work....

rendicott commented 8 years ago

'python pysser.py --list -d -1 -p' should get you debug.

That writes to STDOUT and a pysser.py.log in the same directory.

On Jul 25, 2016 5:05 PM, "power8ce" notifications@github.com wrote:

Yea... I already do have both API and Section settings set correct. I just double-checked again.

I tried creating a new API App id and new user. Still the same.

Is there some debugging mode I can run? I tried supplying argument "-d ?" (? being the numbers according to your doc) it doesn't produce anything more of an error.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rendicott/pySser/issues/1#issuecomment-235084863, or mute the thread https://github.com/notifications/unsubscribe-auth/AGKh74kULFJeeM0FRR89JzPCVY2zdAxeks5qZST4gaJpZM4JRG_0 .

power8ce commented 8 years ago

Looks like it's trying to get to URL http://192.168.7.240/phpipam/CLI/user/ and getting 404 error.

2016-07-26 10:37:32,612:DEBUG:Full path to config file: /usr/local/sbin/pySser/runningconfig.ini 2016-07-26 10:37:32,613:DEBUG:------- ENTERING FUNCTION: dataload() ------- 2016-07-26 10:37:32,613:DEBUG:------- ENTERING FUNCTION: processconfig() ------- 2016-07-26 10:37:32,613:DEBUG:------- Total number of sections = 1 2016-07-26 10:37:32,613:DEBUG:-------_ Looping through sections. Current Section = global 2016-07-26 10:37:32,624:INFO:Starting new HTTP connection (1): 192.168.7.240 2016-07-26 10:37:32,625:DEBUG:"POST /phpipam/CLI/user/ HTTP/1.1" 404 215

power8ce commented 8 years ago

Ok. I'm getting further. I read more into phpIPAM API doc and sees that it needs HTTP authentication.

I'm understanding that in your doc says

username and password : These are pretty self explanatory. Currently pySser has no support for the crypt functionality with the phpIpam API applications.

So username and password in runningconfig.ini is for phpIPAM API HTTP authentication correct?

Now I get this error in debug...

2016-07-26 15:04:39,512:DEBUG:Full path to config file: /usr/local/sbin/pySser/runningconfig.ini 2016-07-26 15:04:39,513:DEBUG:------- ENTERING FUNCTION: dataload() ------- 2016-07-26 15:04:39,513:DEBUG:------- ENTERING FUNCTION: processconfig() ------- 2016-07-26 15:04:39,513:DEBUG:------- Total number of sections = 1 2016-07-26 15:04:39,513:DEBUG:-------_ Looping through sections. Current Section = global 2016-07-26 15:04:39,523:INFO:Starting new HTTP connection (1): 192.168.7.240 2016-07-26 15:04:39,552:DEBUG:"POST /phpipam/pysser/user/ HTTP/1.1" 302 0 2016-07-26 15:04:39,589:DEBUG:"GET /phpipam/?page=login HTTP/1.1" 302 7318 2016-07-26 15:04:39,621:DEBUG:"GET /phpipam/pysser/user/ HTTP/1.1" 302 None 2016-07-26 15:04:39,648:DEBUG:"GET /phpipam/?page=error&section=404 HTTP/1.1" 200 None

Looks like the HTTP authentication is working and getting through. Then it fails somewhere.

rendicott commented 8 years ago

You're getting a 302 HTTP status code back which is a redirect. I'm guessing it's not detecting that you're trying to do your HTTP auth for some reason. For comparison this is how it should look:

Full path to config file: runningconfig2.ini
------- ENTERING FUNCTION: dataload() -------
------- ENTERING FUNCTION: process_config() -------
-_-_-_-_-_-_-_ Total number of sections = 1
-_-_-_-_-_-_-_ Looping through sections. Current Section = global
Starting new HTTP connection (1): ipfreely.contoso.com
"POST /phpipam/api/testing/user/ HTTP/1.1" 200 103
Expired?: False
Token = tbxYSV1AqdAhYtecV_hpiTr7
Expires = 2016-07-26 22:27:04
------- ENTERING FUNCTION: load_sections() -------
Starting new HTTP connection (1): ipfreely.contoso.com
"GET /phpipam/api/testing//sections/ HTTP/1.1" 200 642
------- ENTERING FUNCTION: load_subnets() -------
url_subnets: http://ipfreely.contoso.com/phpipam/api/testing//sections/13/subnets/
Starting new HTTP connection (1): ipfreely.contoso.com
"GET /phpipam/api/testing//sections/13/subnets/ HTTP/1.1" 200 None
------- ENTERING FUNCTION: load_addresses() -------
Starting new HTTP connection (1): ipfreely.contoso.com
"GET /phpipam/api/testing//subnets/329/addresses/ HTTP/1.1" 404 59
Starting new HTTP connection (1): ipfreely.contoso.com
"GET /phpipam/api/testing//subnets/330/addresses/ HTTP/1.1" 404 59
.......

And here's my exact config file:

[global]
base_url: http://ipfreely.contoso.com/phpipam/api/
app_id: testing
username: Admin
password: coolpassw0rd
default_section: Contoso
filter_subnet: 10.11.11.0
hostname_validation: False

I recommend getting an interactive API tester like PostMan (http://www.getpostman.com/) and test the phpIpam API directly to make sure you're able to authenticate and isolate the problem.

Here is how you could auth and query the API with postman

image

image

After you send that POST you can grab the token it sends back: image

Then you can put that token in to GET header and make sure you're authorized. If you get a generic message other than "Unauthorized" it means your username, password, and API app are working.

image

power8ce commented 8 years ago

Ok. I added "/api/" to end so now it's base_url: http://192.168.7.240/phpipam/api/

Making progress!

2016-07-26 15:52:58,240:DEBUG:Full path to config file: /usr/local/sbin/pySser/runningconfig.ini 2016-07-26 15:52:58,240:DEBUG:------- ENTERING FUNCTION: dataload() ------- 2016-07-26 15:52:58,240:DEBUG:------- ENTERING FUNCTION: processconfig() ------- 2016-07-26 15:52:58,240:DEBUG:------- Total number of sections = 1 2016-07-26 15:52:58,240:DEBUG:-------_ Looping through sections. Current Section = global 2016-07-26 15:52:58,250:INFO:Starting new HTTP connection (1): 192.168.7.240 2016-07-26 15:52:58,289:DEBUG:"POST /phpipam/api/pysser/user/ HTTP/1.1" 200 103 2016-07-26 15:52:58,292:INFO:Expired?: False 2016-07-26 15:52:58,292:INFO:Token = Bz%sXhtdR0slLyd!=rCB44f% 2016-07-26 15:52:58,292:INFO:Expires = 2016-07-26 21:52:58 2016-07-26 15:52:58,292:DEBUG:------- ENTERING FUNCTION: load_sections() ------- 2016-07-26 15:52:58,294:INFO:Starting new HTTP connection (1): 192.168.7.240 2016-07-26 15:52:58,294:DEBUG:"GET /phpipam/api/pysser//sections/ HTTP/1.1" 401 500

I get 401 instead of the 200 code you got. I'll check out PostMan. Thanks!

power8ce commented 8 years ago

For some reason I did not needed to enable HTTP authentication on Apache. It's running through but encounter some errors.

Starting new HTTP connection (1): 192.168.7.240 "POST /phpipam/api/pysser/user/ HTTP/1.1" 200 103 Expired?: False Token = riEsh_X.$.wCJCFedRDFbSyG Expires = 2016-07-27 19:09:25 ------- ENTERING FUNCTION: load_sections() ------- Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//sections/ HTTP/1.1" 200 314 ------- ENTERING FUNCTION: load_subnets() ------- url_subnets: http://192.168.7.240/phpipam/api/pysser//sections//subnets/ Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//sections//subnets/ HTTP/1.1" 404 50 ERROR ------- ENTERING FUNCTION: load_addresses() ------- ------- ENTERING FUNCTION: filter_addresses() ------- Exiting filter_addresses with addresses length: '0' ------- ENTERING FUNCTION: group_by_hostname() ------- Made 0 associated_ids modifications to addresses ------- ENTERING FUNCTION: import_addresses() ------- http://192.168.7.240/phpipam/api/pysser//addresses/

------- ENTERING FUNCTION: filter_addresses() ------- Exiting filter_addresses with addresses length: '0'

Number of filtered results: 0 METRIC COUNT

            subnets        0
            addresses      0
            uniquer_iter   1
            cstm_attrs     0
            cstm_fltrs     0

SESSION: default_section_id filter_subnet 192.168.7.0 generator <classes.UniqueGenerator.UniqueGenerator instance at 0x1d107a0> hostname_validation True token riEsh_X.$.wCJCFedRDFbSyG SECTION: DNS None description Default id 3 masterSection 0 name Default order None permission {"2":"2"} permissions None showVLAN 1 showVRF 0 strictMode 1 subnetOrdering default SUBNETS: ERROR: list index out of range ADDRESSES: ERROR: list index out of range

Your load_subnets() outputs is different from mine. Yours: url_subnets: http://ipfreely.contoso.com/phpipam/api/testing//sections/13/subnets/ Mine: url_subnets: http://192.168.7.240/phpipam/api/pysser//sections//subnets/

Looking at my MySQL DB, under Section it's there with id number 3 pysser mysql

It's missing the number. Any thoughts?

rendicott commented 8 years ago

Do you have any subnets or addresses entered into phpIpam for it to load?

power8ce commented 8 years ago

Yes.

pysser ips

power8ce commented 8 years ago

N/M... The default_section name is case sensitive.

power8ce commented 8 years ago

For some reason it's not loading the new IP. Here's what happens...

Command ran: python pysser.py -c runningconfig.ini --addressfile=new_ips.csv -d -1 -p

Content of runningconfig.ini:

[global]
# https is required. All API calls expect it.
#base_url: https://phpipam.mycompany.com/phpipam
base_url: http://192.168.7.240/phpipam/api
# the app_id that you have set up in your phpIpam API applications
#  see: http://phpipam.net/api-documentation/
app_id: pysser
username: test
password: testtest123
# the default section in which to interact with the API
default_section: Default

Content of new_ips.csv 192.168.7.200,TESTTEST,USED,awesome box,00:50:56:bb:66:66 (192.168.7.200 is free and available in phpIPAM)

Starting new HTTP connection (1): 192.168.7.240 "POST /phpipam/api/pysser/user/ HTTP/1.1" 200 103 Expired?: False Token = riEsh_X.$.wCJCFedRDFbSyG Expires = 2016-07-27 20:36:20 ------- ENTERING FUNCTION: load_sections() ------- Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//sections/ HTTP/1.1" 200 331 ------- ENTERING FUNCTION: load_subnets() ------- url_subnets: http://192.168.7.240/phpipam/api/pysser//sections/3/subnets/ Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//sections/3/subnets/ HTTP/1.1" 200 500 ------- ENTERING FUNCTION: load_addresses() ------- Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//subnets/7/addresses/ HTTP/1.1" 200 1504 ------- ENTERING FUNCTION: filter_addresses() ------- Exiting filter_addresses with addresses length: '4' ------- ENTERING FUNCTION: group_by_hostname() ------- Made 0 associated_ids modifications to addresses ------- ENTERING FUNCTION: import_addresses() ------- http://192.168.7.240/phpipam/api/pysser//addresses/

------- ENTERING FUNCTION: filter_addresses() ------- Exiting filter_addresses with addresses length: '4' [] [] True TEST01 TEST01 11 192.168.7.1 None 00:50:56:bb:66: [] 7 2 [] [] None True TEST02 TEST02 12 192.168.7.2 None 00:50:56:bb:66:None None [] 7 2 [] [] None True TEST03 TEST03 13 192.168.7.66 None 00:50:56:bb:22:None None [] 7 2 [] [] None True TEST04 TEST04 14 192.168.7.100 None 00:50:56:bb:99:TEST Admin [] 7 2

Number of filtered results: 4 METRIC COUNT

            subnets        1
            addresses      4
            uniquer_iter   1
            cstm_attrs     0
            cstm_fltrs     0

SESSION: default_section_id 3 filter_subnet None generator <classes.UniqueGenerator.UniqueGenerator instance at 0x101f170> hostname_validation True token riEsh_X.$.wCJCFedRDFbSyG SECTION: DNS None description Default id 3 masterSection 0 name Default order None permission {"2":"3"} permissions None showVLAN 1 showVRF 0 strictMode 1 subnetOrdering default SUBNETS: description id isFolder isFull mask permissions sectionId subnet tag vlanId Default 7 0 0 24 {"2":"2"} 3 192.168.7.0 2 0 ADDRESSES: custom_attributes custom_filters description filter_included hostname hostname_short id ip is_gateway mac note owner related_ids subnetId tag [] [] True TEST01 TEST01 11 192.168.7.1 None 00:50:56:bb:66: [] 7 2 [] [] None True TEST02 TEST02 12 192.168.7.2 None 00:50:56:bb:66:None None [] 7 2 [] [] None True TEST03 TEST03 13 192.168.7.66 None 00:50:56:bb:22:None None [] 7 2 [] [] None True TEST04 TEST04 14 192.168.7.100 None 00:50:56:bb:99:TEST Admin []

rendicott commented 8 years ago

If the subnet for those addresses doesn't exist in phpIpam you need to load the subnets in with a file and the --subnetfile option. I'm not sure if loading addresses without the subnet file works. I didn't really test that functionality. The import functions were quick and dirty.

On Jul 27, 2016 3:42 PM, "power8ce" notifications@github.com wrote:

For some reason it's not loading the new IP. Here's what happens...

Command ran: _python pysser.py -c runningconfig.ini --addressfile=newips.csv -d -1 -p

Content of runningconfig.ini:

[global]

https is required. All API calls expect it.

base_url: https://phpipam.mycompany.com/phpipam

base_url: http://192.168.7.240/phpipam/api

the app_id that you have set up in your phpIpam API applications

see: http://phpipam.net/api-documentation/

app_id: pysser username: test password: testtest123

the default section in which to interact with the API

default_section: Default

Content of new_ips.csv 192.168.7.200,TESTTEST,USED,awesome box,00:50:56:bb:66:66 (192.168.7.200 is free and available in phpIPAM)

Starting new HTTP connection (1): 192.168.7.240 "POST /phpipam/api/pysser/user/ HTTP/1.1" 200 103 Expired?: False Token = riEsh_X.$.wCJCFedRDFbSyG Expires = 2016-07-27 20:36:20

------- ENTERING FUNCTION: load_sections() ------- Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//sections/ HTTP/1.1" 200 331

------- ENTERING FUNCTION: load_subnets() ------- url_subnets: http://192.168.7.240/phpipam/api/pysser//sections/3/subnets/

Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//sections/3/subnets/ HTTP/1.1" 200 500

------- ENTERING FUNCTION: load_addresses() ------- Starting new HTTP connection (1): 192.168.7.240 "GET /phpipam/api/pysser//subnets/7/addresses/ HTTP/1.1" 200 1504

------- ENTERING FUNCTION: filter_addresses() ------- Exiting filter_addresses with addresses length: '4'

------- ENTERING FUNCTION: group_by_hostname() ------- Made 0 associated_ids modifications to addresses ------- ENTERING FUNCTION: import_addresses() -------

http://192.168.7.240/phpipam/api/pysser//addresses/

------- ENTERING FUNCTION: filter_addresses() ------- Exiting filter_addresses with addresses length: '4' [] [] True TEST01 TEST01 11 192.168.7.1 None 00:50:56:bb:66: [] 7 2 [] [] None True TEST02 TEST02 12 192.168.7.2 None 00:50:56:bb:66:None None [] 7 2 [] [] None True TEST03 TEST03 13 192.168.7.66 None 00:50:56:bb:22:None None [] 7 2 [] [] None True TEST04 TEST04 14 192.168.7.100 None 00:50:56:bb:99:TEST Admin [] 7 2

Number of filtered results: 4

METRIC COUNT

subnets 1 addresses 4

uniquer_iter 1 cstm_attrs 0

cstm_fltrs 0

SESSION: default_section_id 3 filter_subnet None

generator hostname_validation True token riEsh_X.$.wCJCFedRDFbSyG SECTION: DNS None description Default id 3 masterSection 0 name Default order None permission {"2":"3"}

permissions None showVLAN 1 showVRF 0 strictMode 1 subnetOrdering default SUBNETS: description id isFolder isFull mask permissions sectionId subnet tag vlanId Default 7 0 0 24 {"2":"2"} 3 192.168.7.0 2 0 ADDRESSES: custom_attributes custom_filters description filter_included hostname hostname_short id ip is_gateway mac note owner related_ids subnetId tag [] [] True TEST01 TEST01 11 192.168.7.1 None 00:50:56:bb:66: [] 7 2 [] [] None True TEST02 TEST02 12 192.168.7.2 None 00:50:56:bb:66:None None [] 7 2 [] [] None True TEST03 TEST03 13 192.168.7.66 None 00:50:56:bb:22:None None [] 7 2 [] [] None True TEST04 TEST04 14 192.168.7.100 None 00:50:56:bb:99:TEST Admin []

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/rendicott/pySser/issues/1#issuecomment-235697150, or mute the thread https://github.com/notifications/unsubscribe-auth/AGKh7yY-b5Y4tZ0nOSxFF87SCgMYfEovks5qZ7SLgaJpZM4JRG_0 .

power8ce commented 8 years ago

I tried. Still not working.

I also tried adding new subnet 192.168.8.0 with one new IP. However, It adds the new subnet in phpIPAM but not the IP.

# cat new_subnets.csv
subnet
192.168.8.0/24
# cat new_ips.csv
192.168.8.200,TESTTEST,USED,awesome box,00:50:56:bb:66:66

Command ran: python pysser.py -c runningconfig.ini --subnetfile=new_subnets.csv --addressfile=new_ips.csv -d -1 -p

power8ce commented 8 years ago

Ok! It's all working now!

It needed the header line in for addressfile.

So my new_ips.csv looks like this and it's working!

Ip,Name,Metadata,Description,mac
192.168.8.1,TEST88,USED,awesome box,00:50:56:bb:96:51

Thank you!