Closed sraustein closed 11 years ago
{{{ $ ssh ca0.vmini.rpki.net ssh: connect to host ca0.vmini.rpki.net port 22: No route to host }}}
If I understand the problem, user attempted to create a ROA for AS4128, and the prefixes you expected to automatically be detected were not. But did work as expected for AS3130?
Trac comment by melkins on 2013-07-26T15:48:37Z
If I understand the problem, user attempted to create a ROA for AS4128, and the prefixes you expected to automatically be detected were not.
i.e. the "this will happen" only showed 3130, not the 4128 announcement
a number of labuserNNs had this problem
But did work as expected for AS3130?
yes
randy
Trac comment by randy on 2013-07-26T19:16:22Z
i.e. the "this will happen" only showed 3130, not the 4128 announcement
a number of labuserNNs had this problem
But did work as expected for AS3130?
The "this will happen" table only shows changes. Was there already a ROA for AS4128 making that route valid? Because in that case creation of the ROA for AS3130 won't affect the other route, and therefore it would not be displayed as currently implemented.
Trac comment by melkins on 2013-07-26T22:16:11Z
i.e. the "this will happen" only showed 3130, not the 4128 announcement
a number of labuserNNs had this problem
But did work as expected for AS3130?
The "this will happen" table only shows changes. Was there already a ROA for AS4128 making that route valid? Because in that case creation of the ROA for AS3130 won't affect the other route, and therefore it would not be displayed as currently implemented.
there were no pre-existing roas. sra had wiped the labusers
randy
Trac comment by randy on 2013-07-27T05:47:15Z
there were no pre-existing roas. sra had wiped the labusers
How long before this problem occurred were the labusers wiped?
Just speculating that it is possible that the GUI had not yet picked up the fact that an old ROA was deleted because rcynic had not run after the object was revoked and pubd removed it.
Trac comment by melkins on 2013-07-29T16:25:02Z
How long before this problem occurred were the labusers wiped?
as i remember, a few hours
Trac comment by randy on 2013-07-29T16:28:23Z
there were no pre-existing roas. sra had wiped the labusers
How long before this problem occurred were the labusers wiped?
At least six hours, I think. I forget the exact timing, because we were having far too much fun with bad network configuration on vmini, but I whacked the labuser content (at the CLI level -- rpkic load_foo /dev/null) sometime Thursday evening my time, so call it no later than 04:00Z, and the workshop didn't start until noon in Berlin, 10:00Z.
Just speculating that it is possible that the GUI had not yet picked up the fact that an old ROA was deleted because rcynic had not run after the object was revoked and pubd removed it.
rcynic was running every 2 minutes, although the aforementioned network problem may have prevented it from working properly until around 02:00Z when I whacked vmini's ICMP redirect cache.
Trac comment by sra on 2013-07-29T16:37:32Z
bump
this happened again in xi'an, and to four students. this is now the most broken thing in the workshop day.
Trac comment by randy on 2013-09-09T15:58:51Z
On 09/09/2013 08:58 AM, Trac Ticket System wrote:
588: all origin ASs not showing up in ROA confirm
-----------------------+---------------------- Reporter: randy | Owner: melkins Type: defect | Status: accepted Priority: critical | Component: gui Resolution: | Keywords: Blocked By: | Blocking: -----------------------+----------------------
Comment (by randy):
bump
this happened again in xi'an, and to four students. this is now the most broken thing in the workshop day.
is this on vmini? can you add my ssh key so i can take a look?
Trac comment by melkins on 2013-09-09T16:02:35Z
is this on vmini? can you add my ssh key so i can take a look?
done
Trac comment by randy on 2013-09-09T16:09:12Z
On 09/09/2013 08:58 AM, Trac Ticket System wrote:
588: all origin ASs not showing up in ROA confirm
-----------------------+---------------------- Reporter: randy | Owner: melkins Type: defect | Status: accepted Priority: critical | Component: gui Resolution: | Keywords: Blocked By: | Blocking: -----------------------+----------------------
Comment (by randy):
bump
this happened again in xi'an, and to four students. this is now the most broken thing in the workshop day.
Not sure how to ssh into ca0.vmini from vmini since it is asking for root's password.
Can you make sure you are not pulling in ROAs from altCA, because I do see the following ROAs for AS4128, which if included in your set would make the ROA confirmation page not display those routes, since it only displays /changes/ to route validity.
{{{ In [17]: [r.prefix_min for r in ROAPrefixV4.objects.filter(roas__asid=4128)] Out[17]: [<rpki.POW.IPAddress object 98.128.1.0 at 0xae835c0>, <rpki.POW.IPAddress object 98.128.1.0 at 0xae830c0>, <rpki.POW.IPAddress object 98.128.2.0 at 0xae83980>, <rpki.POW.IPAddress object 98.128.2.0 at 0xae83a00>, <rpki.POW.IPAddress object 98.128.3.0 at 0xae83a80>, <rpki.POW.IPAddress object 98.128.4.0 at 0xae83b00>, <rpki.POW.IPAddress object 98.128.5.0 at 0xae83b80>, <rpki.POW.IPAddress object 98.128.6.0 at 0xae83c00>, <rpki.POW.IPAddress object 98.128.8.0 at 0xae83c80>, <rpki.POW.IPAddress object 98.128.8.0 at 0xae83d00>, <rpki.POW.IPAddress object 98.128.9.0 at 0xae83d80>, <rpki.POW.IPAddress object 98.128.10.0 at 0xae83e00>, <rpki.POW.IPAddress object 98.128.11.0 at 0xae83e80>, <rpki.POW.IPAddress object 98.128.12.0 at 0xae83f00>, <rpki.POW.IPAddress object 98.128.14.0 at 0xae83f80>, <rpki.POW.IPAddress object 98.128.15.0 at 0xae83ec0>, <rpki.POW.IPAddress object 98.128.30.0 at 0xae83fc0>, <rpki.POW.IPAddress object 147.28.224.0 at 0xae83400>, <rpki.POW.IPAddress object 198.180.152.0 at 0xae83660>] }}}
Trac comment by melkins on 2013-09-09T17:00:43Z
Can you make sure you are not pulling in ROAs from altCA,
ca0.vmini.rpki.net most definitely //does// pull from altCA, as that's where its own outputs are published.
Trac comment by sra on 2013-09-09T19:49:23Z
Can you make sure you are not pulling in ROAs from altCA,
ca0.vmini.rpki.net most definitely //does// pull from altCA, as that's where its own outputs are published.
Properly speaking, s/altCA/ca0.rpki.net/.
Trac comment by sra on 2013-09-09T19:50:34Z
On 09/09/2013 12:50 PM, Trac Ticket System wrote:
Can you make sure you are not pulling in ROAs from altCA,
ca0.vmini.rpki.net most definitely //does// pull from altCA, as that's where its own outputs are published.
Properly speaking, s/altCA/ca0.rpki.net/.
The following ROAs for AS4128 exist:
{{{
}}}
Creating a ROA for AS3130 will not invalidate the AS4128 ROAs, therefore the GUI excludes the routes via AS4128 that are already valid from the ROA confirmation page because they will not change.
I must be missing something, or not understanding the question...
Trac comment by melkins on 2013-09-09T20:00:17Z
Randy will have to supply details, but my notes from remote support of Xi'an say that:
We did not see any pattern we understood to which labuserNN accounts were affected.
Trac comment by sra on 2013-09-09T20:12:20Z
On 09/09/2013 01:12 PM, Trac Ticket System wrote:
588: all origin ASs not showing up in ROA confirm
-----------------------+---------------------- Reporter: randy | Owner: melkins Type: defect | Status: accepted Priority: critical | Component: gui Resolution: | Keywords: Blocked By: | Blocking: -----------------------+----------------------
Comment (by sra):
Randy will have to supply details, but my notes from remote support of Xi'an say that:
- The routeviews data itself was OK when I checked it (contained the expected routes);
The problem of not showing the routes that would be affected by RPKI changes hit //some// but not //all// labuserNN accounts on ca0.vmini.rpki.net.
We did not see any pattern we understood to which labuserNN accounts were affected.
I would expect different behavior because there are not ROAs for AS4128 for all labuser accounts (there are not ROAs covering 98.128.7.0/24, 98.128.13.0/24), so for those accounts the AS4128 route would should up in the ROA conformation page as becoming invalid)
Trac comment by melkins on 2013-09-09T20:16:52Z
I think it would be helpful to give me the script that is followed during the lab exercise. Something like:
0) initial state is all ROAs removed from labuser accounts 1) view routes tab in gui to see routes via AS3130 and AS4128, both UNKNOWN 2) create ROA for AS3130, note in the confirmation page that AS3130 will go VALID, while AS4128 goes INVALID 3) create ROA for AS4128, note that the AS4128 route goes VALID (and the route via AS3130 is unaffected)
Trac comment by melkins on 2013-09-09T20:25:09Z
I would expect different behavior because there are not ROAs for AS4128 for all labuser accounts (there are not ROAs covering 98.128.7.0/24, 98.128.13.0/24), so for those accounts the AS4128 route would should up in the ROA conformation page as becoming invalid)
We may still not be communicating. At the risk of tedious repetition, the scenario as I understand it is:
We do not understand why student A gets the warning and student B does not. We think that both students should get the warning. We suspect this is a bug.
In Xi'an, there were four cases of student B, and it was fairly embarrassing. The workaround, if I understood correctly, was to tell all the B students to pick different (unused) student numbers and try again with the new accounts, so while it's theoretically possible that this is some kind of subtle pilot error that we didn't see, it's not likely, as the same students got it right with the new accounts.
If that's not clear enough, it'll have to wait for Randy. Likewise, if I've garbled the story, I expect Randy will straighten it out.
Trac comment by sra on 2013-09-09T20:32:25Z
Not sure how to ssh into ca0.vmini from vmini since it is asking for root's password.
ssh in directly
ssh -p 42043 vmini.rpki.net
Trac comment by randy on 2013-09-09T23:49:50Z
I think it would be helpful to give me the script that is followed during the lab exercise. Something like:
0) initial state is all ROAs removed from labuser accounts
yes
1) view routes tab in gui to see routes via AS3130 and AS4128, both UNKNOWN
no
2) create ROA for AS3130, note in the confirmation page that AS3130 will go VALID, while AS4128 goes INVALID
that is the intent, except o the asns could be reversed o the confirmation page may or may not show one or both of the asns, which is the bug
3) create ROA for AS4128, note that the AS4128 route goes VALID (and the route via AS3130 is unaffected)
i am not sure what your intent is here. but what i would like to see is all announcements covered by the prefix, period, whether altered or not
randy
Trac comment by randy on 2013-09-09T23:53:45Z
- Students sit down at their shiny new wiped clean GUI accounts. Each student has a /24 delegated by the workshop parent user. Nobody has ROAs because all the ROAs were wiped before the class starts. Everybody has routes in BGP, and the raw routeviews data shows them.
note that it is the same bgp cache for all students
- Student A tries to create a ROA using her prefix, GUI warns her that there are routes for that prefix and what they are.
Student B tries to create a ROA using her prefix, GUI does //not// warn her that there are routes for that prefix or what they are.
We do not understand why student A gets the warning and student B does not. We think that both students should get the warning. We suspect this is a bug.
that is it
In Xi'an, there were four cases of student B, and it was fairly embarrassing. The workaround, if I understood correctly, was to tell all the B students to pick different (unused) student numbers and try again with the new accounts, so while it's theoretically possible that this is some kind of subtle pilot error that we didn't see, it's not likely, as the same students got it right with the new accounts.
correct
Trac comment by randy on 2013-09-09T23:55:22Z
i am not sure what your intent is here. but what i would like to see is all announcements covered by the prefix, period, whether altered or not
original use case was "show me what will change". routes that are covered by your new ROA but which are already valid won't show up since you can't affect the validity.
if you want to see all covered routes, regardless of whether the validity will change, i can do that.
Trac comment by melkins on 2013-09-09T23:57:53Z
On 09/09/2013 04:49 PM, Trac Ticket System wrote:
ssh -p 42043 vmini.rpki.net {{{ $ ssh -p 42043 vmini.rpki.net ssh: connect to host vmini.rpki.net port 42043: Connection refused }}}
Trac comment by melkins on 2013-09-10T00:02:23Z
if you want to see all covered routes, regardless of whether the validity will change, i can do that.
i think so.
but now that you mention it, it might be good to know if other roas are out there in my space. though this is kinda separate. and next, i am gonna wonder who issued them. and the rpki is so nice about the 'who' thing :(
Trac comment by randy on 2013-09-10T00:06:09Z
sorry
{{{ Host ca0.vmini* Hostname vmini.rpki.net Port 42022 ControlMaster no
Host cache0.vmini* Hostname vmini.rpki.net Port 42023 ControlMaster no }}}
Trac comment by randy on 2013-09-10T00:06:50Z
but now that you mention it, it might be good to know if other roas are out there in my space. though this is kinda separate. and next, i am gonna wonder who issued them. and the rpki is so nice about the 'who' thing :(
currently you can view this info from the 'route view' page. click on the validity status and it will show you a list of all ROAs that cover that route (yours and any of your RPKI ancestors). you can click on a ROA from that list to view the detail, which at least gives you a hint in the form of ghostbuster and object URI.
Trac comment by melkins on 2013-09-10T02:04:39Z
but now that you mention it, it might be good to know if other roas are out there in my space. though this is kinda separate. and next, i am gonna wonder who issued them. and the rpki is so nice about the 'who' thing :(
currently you can view this info from the 'route view' page. click on the validity status and it will show you a list of all ROAs that cover that route (yours and any of your RPKI ancestors). you can click on a ROA from that list to view the detail, which at least gives you a hint in the form of ghostbuster and object URI.
if i have a lot of routes, the routes 'page' gets tough. could i have a button next to a resource on Dashboard/Resources to get me to a list of all covering roas and their match status?
Trac comment by randy on 2013-09-10T02:33:07Z
My new script showed the routes for labuser06 as such: {{{ melkins@disaster:/usr/local/src/rpki/trunk/rpkid/portal-gui/scripts$ ./rpkigui-query-routes 98.128.6.0 98.128.0.0/16 4128 unknown 98.128.0.0/16 3130 unknown 98.128.6.0/24 4128 valid 98.128.6.0/24 3130 rsync://rgnet.rpki.net/rpki/rgnet/labuser06/717/gg2_XbyCV63WKD7fxwGLZFlNZXg.roa 98.128.6.0/24 4128 rsync://ca0.rpki.net/rpki/altCA/workshop/labuser06/8/p2xdLD0iAtTCoX7lL-I7hqmZsFg.roa 98.128.6.0/24 3130 valid 98.128.6.0/24 3130 rsync://rgnet.rpki.net/rpki/rgnet/labuser06/717/gg2_XbyCV63WKD7fxwGLZFlNZXg.roa 98.128.6.0/24 4128 rsync://ca0.rpki.net/rpki/altCA/workshop/labuser06/8/p2xdLD0iAtTCoX7lL-I7hqmZsFg.roa }}}
But I noticed that the roa object was not actually in the rcynic cache {{{ melkins@disaster:/usr/local/src/rpki/trunk/rpkid/portal-gui/scripts$ ls -l /var/rcynic/data/authenticated/ca0.rpki.net/rpki/altCA/workshop/labuser06/8/ total 8 -rw-rw-r-- 1 rcynic rcynic 598 Sep 19 13:20 thjbtncUrc8_fXWeR6y-QjlcK74.crl -rw-rw-r-- 1 rcynic rcynic 1850 Sep 19 13:20 thjbtncUrc8_fXWeR6y-QjlcK74.mft }}}
After looking at rcynic.xml I see this:
{{{ melkins@disaster:/usr/local/src/rpki/trunk/rpkid/portal-gui/scripts$ grep p2xdLD0iAtTCoX7lL-I7hqmZsFg.roa /var/rcynic/data/rcynic.xml
}}}
I believe what has happened here is that my garbage collection mechanism is not quite correct. That roa likely was accepted at some point in the past, but fell out of the manifest when it was forgotten/revoked. But the way I garbage collect is to look for objects that had zero mentions in the rcynic.xml, which is not the case for the above ROA. Now, I only parse the roa when I see object_accepted, so when the object falls out of the manifest, the parsed data structure is not removed since it still has a validation status on it!
Trac comment by melkins on 2013-09-19T21:34:36Z
Note that skipped_because_not_in_manifest
is a relatively new status code, which may be why you didn't notice this until now.
Trac comment by sra on 2013-09-19T22:27:30Z
In [changeset:5502]: {{{
delete existing objects that were previously accepted if they were not accepted during the most recent run.
see #588
refactor much of the code in process_cache() into save_statuses() to make it simpler to handle the garbage collection. first we collected all statuses, then save them all at once. }}}
Trac comment by melkins on 2013-09-19T22:53:57Z
With the fix from [5502], the skipped ROA is no longer present in the database:
{{{ melkins@disaster:/usr/local/src/rpki/trunk/rpkid/portal-gui/scripts$ ./rpkigui-query-routes 98.128.6.0 98.128.0.0/16 4128 unknown 98.128.0.0/16 3130 unknown 98.128.6.0/24 4128 invalid 98.128.6.0/24 3130 rsync://rgnet.rpki.net/rpki/rgnet/labuser06/717/gg2_XbyCV63WKD7fxwGLZFlNZXg.roa 98.128.6.0/24 3130 valid 98.128.6.0/24 3130 rsync://rgnet.rpki.net/rpki/rgnet/labuser06/717/gg2_XbyCV63WKD7fxwGLZFlNZXg.roa }}}
I believe this should solve the problem. Phew!
Trac comment by melkins on 2013-09-19T22:57:44Z
upgrading and testing now
Trac comment by randy on 2013-09-19T23:23:46Z
https://ca0.vmini.rpki.net/rpki/
{{{ rpki.net 0.5502 Home Help Internal Server Error Whoops! The administrator has been notified of this error. }}}
and a bunch of email of the form
{{{ Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 92, in get_response response = middleware_method(request)
File "/usr/lib/python2.7/dist-packages/django/middleware/common.py", line 57, in process_request host = request.get_host()
File "/usr/lib/python2.7/dist-packages/django/http/request.py", line 72, in get_host "Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): %s" % host)
SuspiciousOperation: Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): ca0.vmini.rpki.net
<WSGIRequest path:/rpki/conf/export, GET:<QueryDict: {}>, POST:<QueryDict: {}>, COOKIES:{'csrftoken': 'rQvdmJbo30ufdinQxXtArEkxDD2Swmmr', 'sessionid': '1e3ae1d64d81c503447b5b2c9ecc1297'}, META:{'DOCUMENT_ROOT': '/var/www', 'GATEWAY_INTERFACE': 'CGI/1.1', 'HTTPS': '1', 'HTTPACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,/_;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip,deflate,sdch', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.8', 'HTTP_CONNECTION': 'keep-alive', 'HTTP_COOKIE': 'sessionid=1e3ae1d64d81c503447b5b2c9ecc1297; csrftoken=rQvdmJbo30ufdinQxXtArEkxDD2Swmmr', 'HTTP_DNT': '1', 'HTTP_HOST': 'ca0.vmini.rpki.net', 'HTTP_REFERER': 'https://ca0.vmini.rpki.net/rpki/', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.65 Safari/537.36', 'PATH_INFO': u'/rpki/conf/export', 'PATH_TRANSLATED': '/usr/share/rpki/wsgi/rpki.wsgi/rpki/conf/export', 'QUERY_STRING': '', 'REMOTE_ADDR': '10.0.0.1', 'REMOTE_PORT': '52377', 'REQUEST_METHOD': 'GET', 'REQUEST_URI': '/rpki/conf/export', 'SCRIPT_FILENAME': '/usr/share/rpki/wsgi/rpki.wsgi', 'SCRIPT_NAME': u'', 'SERVER_ADDR': '10.0.0.2', 'SERVER_ADMIN': 'webmaster@localhost', 'SERVER_NAME': 'ca0.vmini.rpki.net', 'SERVER_PORT': '443', 'SERVER_PROTOCOL': 'HTTP/1.1', 'SERVER_SIGNATURE': '
Apache/2.2.22 (Ubuntu) Server at ca0.vmini.rpki.net Port 443\n', 'SERVER_SOFTWARE': 'Apache/2.2.22 (Ubuntu)', 'SSL_TLS_SNI': 'ca0.vmini.rpki.net', 'mod_wsgi.application_group': 'ca0.vmini.rpki.net|', 'mod_wsgi.callable_object': 'application', 'mod_wsgi.handler_script': '', 'mod_wsgi.input_chunked': '0', 'mod_wsgi.listener_host': '', 'mod_wsgi.listener_port': '443', 'mod_wsgi.process_group': '', 'mod_wsgi.request_handler': 'wsgi-script', 'mod_wsgi.script_reloading': '1', 'mod_wsgi.version': (3, 3), 'wsgi.errors': <mod_wsgi.Log object at 0xa8b11098>, 'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter object at 0xb6f29d10>, 'wsgi.input': <mod_wsgi.Input object at 0xb6ef0278>, 'wsgi.multiprocess': True, 'wsgi.multithread': True, 'wsgi.run_once': False, 'wsgi.url_scheme': 'https', 'wsgi.version': (1, 1)}> }}}Trac comment by randy on 2013-09-19T23:40:24Z
"Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): %s" %
That may be the Django upgrade.
ALLOWED_HOSTS
is controlled by the allowed_hosts
variable in the
[web_portal]
section of your rpki.conf
.
I'd be a little surprised if that were triggering for ca0.vmini.rpki.net, since as far as I know it does operate under its own hostname rather than something else, but who knows.
Trac comment by sra on 2013-09-19T23:48:52Z
That may be the Django upgrade.
ALLOWED_HOSTS
is controlled by theallowed_hosts
variable in the[web_portal]
section of yourrpki.conf
.
Yes, this is a result of the upgrade to Django 1.5. Unrelated to this bug.
Trac comment by melkins on 2013-09-19T23:49:59Z
added
{{{ allowed-hosts = ca0.vmini.rpki.net }}}
to /etc/rpki.conf
restarted rpki-ca service
still blows the same chunks
Trac comment by randy on 2013-09-19T23:58:10Z
service apache2 restart
Trac comment by melkins on 2013-09-20T00:06:15Z
{{{ ca0.vmini.rpki.net:/root# service apache2 restart
but it works!
Trac comment by randy on 2013-09-20T00:08:58Z
the roa create confirmation page seems to show me what i thought it would. yay!
Trac comment by randy on 2013-09-20T00:20:04Z
apache2: Could not reliably determine the server's fully qualified domain name, using ca0.vmini.rpki.net for ServerName
I think that's Apache for "I'm having a bad DNS day".
Trac comment by sra on 2013-09-20T00:26:16Z
ohhhhh. i don't blame it, i guess
{{{ ca0.vmini.rpki.net:/root# host ca0.vmini.rpki.net ca0.vmini.rpki.net has address 10.0.0.2 ca0.vmini.rpki.net is an alias for vmini.rpki.net. ca0.vmini.rpki.net is an alias for vmini.rpki.net. ca0.vmini.rpki.net:/root# host 10.0.0.2 2.0.0.10.in-addr.arpa domain name pointer ca0.vmini.rpki.net. }}}
my aging dns fu tells me that one should not have a cname and other data
Trac comment by randy on 2013-09-20T00:36:16Z
my aging dns fu tells me that one should not have a cname and other data
Correct.
I suspect you're seeing the result of dnsmasq on vmini trying to merge its /etc/hosts with what it sees in the real DNS.
Test for this would be to change the CNAMEs for {ca0,cache0}.vmini in the real DNS to copies of vmini's address records then restart Apache and see if the problem goes away.
dnsmasq may have some way to tweak this merging behavior, it has a rather lengthy configuration file these days. I'm off to the dairy for milk and ice cream, but can check when I get back if you like.
Trac comment by sra on 2013-09-20T00:44:51Z
Test for this would be to change the CNAMEs for {ca0,cache0}.vmini in the real DNS to copies of vmini's address records
did that
then restart Apache and see if the problem goes away.
waiting for dns to settle
Trac comment by randy on 2013-09-20T00:46:29Z
Moving vmini DNS whackiness discussion to new ticket #621.
Trac comment by sra on 2013-09-20T02:06:03Z
Closed with resolution fixed
on ca0.vmini
The following prefixes did NOT show up in the dashboard -> create ROA for AS4128 (only 3130):
98.128.5.0/24 98.128.6.0/24
Trac ticket #588 component gui priority critical, owner melkins, created by randy on 2013-07-26T14:19:57Z, last modified 2013-09-23T21:46:30Z