search

daevaorn / djapian

High level Xapian integration for Django
Other
6 stars 3 forks source link

no able to search all entries. #75

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
When trying to search all the data in the table i am un able it only allows
me to search threw a few rows. 

I am running Django 1.0, Djapian SVN, and Xapian what ever I got with sudo
apt-get. 

My models are set up like this:
#abstract tables
class Alert(models.Model):
    actionReason=models.ForeignKey(Reason)
    DNS=models.CharField(max_length=255,null=True)
    action=models.ForeignKey(Action)
    actionDetail=models.CharField(max_length=255,null=True)
    entryDate=models.DateTimeField(auto_now_add=True)
    actionStart=models.DateTimeField()
    site=models.ForeignKey(Sites)
    addrType=models.ForeignKey(AddrType,null=True)
    nodeType=models.CharField(max_length=50,null=True)
    handling=models.ForeignKey(Handler,null=True)
    file=models.ForeignKey(File)

    def get_fields(self):
        return ['actionReason','DNS',
                'action','actionDetail',
                'entryDate','actionStart',
                'site','addrType',
                'nodeType','handling',
                'file']

    class Meta:
        abstract = True

#table for storing reported malicious IPs and domains
class BadActor(Alert):
    actionDuration=models.PositiveIntegerField(null=True)
    addr=models.CharField(max_length=40,null=True)
    port=models.IntegerField(null=True)
    protocol=models.ForeignKey(Protocol,null=True)
    TLD=models.CharField(max_length=255)
    threshold=models.IntegerField()
    priors=models.IntegerField()
    latitude=models.DecimalField(max_digits=7, decimal_places=4, null=True)
    longitude=models.DecimalField(max_digits=7, decimal_places=4, null=True)

    def get_fields(self):
        lst = super(BadActor, self).get_fields();
        lst.extend(['actionDuration','abbr',
                    'port','protocol',
                    'TLD','threshold',
                    'priors','latitude',
                    'longitude']);
        return lst;

class BadActorIndexer(Indexer):
    fields = ['actionDuration']
    tags = [
        ('actionDuration', 'actionDuration'),
        ('addr', 'addr'),
        ('protocol', 'protocol'),
        ('DNS','DNS'),
    ]

space.add_index(BadActor, BadActorIndexer, attach_as='indexer')

complete_indexer = CompositeIndexer(BadActor.indexer)

END OF MODEL
My View looks like:

from django import forms
from federated.repository.models import BadActor
from federated.repository.models import complete_indexer

MODEL_MAP = {
    'BadActor': BadActor,

}

MODEL_CHOICES = [('', 'all')] + zip(MODEL_MAP.keys(), MODEL_MAP.keys
())

class SearchForm(forms.Form):
    query = forms.CharField(required=True)
    model = forms.ChoiceField(choices=MODEL_CHOICES, required=False)

def repoSearchAlert(request):
    adminuser = after_login(request, 'user')
    logos = getLogos()

    errlst = []
    msglst = []
    results = []

    if( request.POST ):
        form = SearchForm(request.POST)
        if form.is_valid():
            query = form.cleaned_data['query']
            model = MODEL_MAP.get(form.cleaned_data['model'])

            if not model:
                indexer = complete_indexer
            else:
                indexer = model.indexer

            results = indexer.search(query).prefetch()
    else:
        form = SearchForm()

    return render_to_response('repoSearch.html',
                            {'session':request.session,
                             'msg':"<br>".join(msglst),
                             'errmsg':"<br>".join(errlst),
                             'logged_in': request.session.get
('_auth_user_id', False),
                             'logos': logos,
                             'form':form,
                             'results':results,
                            })
repoSearchAlert=login_required(repoSearchAlert)

END OF VIEW

now when I try to search for something it does not work. what I have tried
is located at 
http://groups.google.com/group/djapian-dev/browse_thread/thread/1bf89e5dc215f06b
/85a50022e1d41370?hl=en#85a50022e1d41370

PLEASE HELP I REALLY WANT THIS TO WORK.

Original issue reported on code.google.com by crash....@gmail.com on 1 Sep 2009 at 1:23

GoogleCodeExporter commented 9 years ago 
I for got data so here is some data Sorry for the uglyness i had to blank 
things out:

1   1   Pending 1   Blocked for 86400 seconds   2009-08-14 11:00:03 2009-08-14
10:45:08        1   1   dns     X.X.X.X     22  1   location    185 0   13.75   100.5167    1
2   1   Pending 1   Blocked for 86400 seconds   2009-08-14 11:00:03 2009-08-14
10:45:08        1   1   dns     X.X.X.X     443 1   location    255 0   43.9    -78.6833    1
3   1   Pending 1   Blocked for 86400 seconds   2009-08-14 11:00:03 2009-08-14
10:47:17        1   1   dns     X.X.X.X     22  1   location    185 0   39.9289 116.3883    1
4   1   Pending 1   Blocked for 86400 seconds   2009-08-14 11:00:03 2009-08-14
10:54:08        1   1   dns     X.X.X.X     22  1   location    185 0   22.5333 114.1333    1
5   1   Pending 1   Blocked for 86400 seconds   2009-08-14 11:30:03 2009-08-14
11:24:16        1   1   dns     X.X.X.X     25  1   location    25  0   25.0392 121.525 3
6   1   Pending 1   Blocked for 86400 seconds   2009-08-14 11:30:03 2009-08-14
11:13:20        1   1   dns     X.X.X.X             location    25  27  39.9289 116.3883    2
7   1   Pending 1   Blocked for 86400 seconds   2009-08-14 12:15:02 2009-08-14
12:04:17        1   1   dns     X.X.X.X     22  1   location    185 1   55.7522 37.6156 6
8   1   Pending 1   Blocked for 86400 seconds   2009-08-14 12:15:02 2009-08-14
12:07:18        1   1   dns     X.X.X.X     22  1   location    185 0   51  9   6
9   1   Pending 1   Blocked for 86400 seconds   2009-08-14 12:15:02 2009-08-14
11:49:17        1   1   dns     X.X.X.X     22  1   location    185 0   41.725  44.7908 5
10  1   Pending 1   Blocked for 86400 seconds   2009-08-14 12:30:03 2009-08-14
12:22:17        1   1   dns     X.X.X.X     22  1   location    185 2   39.9289 116.3883    7
11  1   Pending 1   Blocked for 86400 seconds   2009-08-14 13:00:03 2009-08-14
12:56:08        1   1   dns     X.X.X.X     8080    1   location    25  0   39.9272 32.8644 9
12  1   Pending 1   Blocked for 86400 seconds   2009-08-14 13:25:25 2009-08-14
13:08:17        1   1   dns     X.X.X.X     8080    1   location    25  3   36.6683 116.9972    10
13  1   Pending 1   Blocked for 86400 seconds   2009-08-14 13:25:25 2009-08-14
13:13:08        1   1   dns     X.X.X.X     8080    1   location    25  3   31.005  121.4086    10
14  1   Pending 1   Blocked for 86400 seconds   2009-08-14 13:25:25 2009-08-14
13:13:47        1   1   dns     X.X.X.X             location    25  330 22.2833 114.15  10
15  1   Pending 1   Blocked for 86400 seconds   2009-08-14 13:26:03 2009-08-13
14:14:11        1   1   dns     X.X.X.X             location    25  554 37.5664 126.9997    11

Original comment by crash....@gmail.com on 1 Sep 2009 at 1:29

GoogleCodeExporter commented 9 years ago

Original comment by daevaorn on 1 Sep 2009 at 1:47