Allow OSINT Sources to Be Customized

[magneticstain]

Right now, augur uses a hard coded list of OSINT sources.

# GLOBALS
# set OSINT API URLs
OSINT_API_URLS = {
    'SANS_DShield': 'https://isc.sans.edu/api/openiocsources/'
}

This is an issue since the URL or name can change at any time. And because we should allow users to add any sources they choose.

To do this, we will need to have a way to:

1. Set source URLs in the database
2. Have a way to specify the XML layout so that Augur can properly parse it.

The first step is easy, the second will be where I suspect things will get complicated.

Breaks down to:

• [x] Create new table in Inquisition DB for sources (IOCSources)
• [x] Update Tuning API to return/support IOC sources
• [ ] Add support for listing and editing sources in Intel section on Tuning section of UI
• [ ] Update Augur to support reading from multiple sources and creating threat records from each

[magneticstain]

MariaDB [inquisition]> show create table IOCSources;
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table      | Create Table                                                                                                                                                                                                                                                                                                                                  |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| IOCSources | CREATE TABLE `IOCSources` (
  `source_id` int(11) NOT NULL AUTO_INCREMENT,
  `created` datetime DEFAULT NULL,
  `updated` datetime DEFAULT NULL,
  `source_type` varchar(10) DEFAULT NULL,
  `source_name` varchar(30) DEFAULT NULL,
  `endpoint` varchar(120) DEFAULT NULL,
  PRIMARY KEY (`source_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MariaDB [inquisition]>