omz / AppSales-Mobile

App Sales allows iPhone and Mac App Store developers to download and analyze their daily and weekly sales reports from iTunes Connect.
http://omz-software.com
1.89k stars 407 forks source link

Make Customer review download over RSS #237

Open abuharsky opened 11 years ago

abuharsky commented 11 years ago

1) add JSONKit 2) Change "start" method in ReviewDownloadManager.m to

- (void)start
{
    backgroundTaskID = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:nil];

    NSString *productID = _product.productID;
    NSString *URLString = [NSString stringWithFormat:@"https://itunes.apple.com/%@/rss/customerreviews/id=%@/sortBy=mostRecent/json", country, productID];
    NSURL *URL = [NSURL URLWithString:URLString];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:URL];

    self.downloadConnection = [NSURLConnection connectionWithRequest:request delegate:self];
}

3) Change this lines in "connectionDidFinishLoading"

- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    NSString *html = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding] autorelease];
    if (html)
    {
        dispatch_async(dispatch_get_global_queue(0, 0), ^ {

            NSArray *reviewInfos = [self reviewInfosFromHTML:html];

to

- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    if(decoder == nil)
        decoder = [[JSONDecoder alloc] initWithParseOptions:JKParseOptionLooseUnicode];

    NSDictionary *response = [decoder objectWithData:data];
    NSArray *entry = [response valueForKeyPath:@"feed.entry"];

    if (entry.count > 0)
    {
        dispatch_async(dispatch_get_global_queue(0, 0), ^ {

           NSMutableArray *reviewInfos = [NSMutableArray new];

            for(NSDictionary *info in entry)
            {
                if([info objectForKey:@"author"] != nil && [info objectForKey:@"content"] != nil)
                {
                    NSDictionary *reviewInfo = [NSDictionary dictionaryWithObjectsAndKeys:
                                                [info valueForKeyPath:@"title.label"], kReviewInfoTitle,
                                                [info valueForKeyPath:@"author.name.label"], kReviewInfoUser,
                                                [info valueForKeyPath:@"im:version.label"], kReviewInfoVersion,
//                                                [info valueForKeyPath:@"title.label"], kReviewInfoDateString,
                                                [info valueForKeyPath:@"content.label"], kReviewInfoText,
                                                [NSNumber numberWithInt:[[info valueForKeyPath:@"im:rating.label"] intValue]], kReviewInfoRating,
                                                nil];
                    [reviewInfos addObject:reviewInfo];
                }
            }
wayneljw commented 11 years ago

Thanks for the instruction. 2 more things need to be done in ReviewDownloadManager.h

at the beginning, add:

#import "JSONKit.h"

inside "@interface ReviewDownload : NSObject {}", add:

JSONDecoder * decoder;
AlessandroPappalardo commented 11 years ago

Perfect! The download works correctly. The only problem is that the reviews have the date set to null. I noticed that the date doesn't exists in the RSS (tag "updated"). There is an easy way to fix this?

AlessandroPappalardo commented 11 years ago

Now I can correctly extract the dates of the reviews! I solved reimplementing all with an XML parser without json. Of course I fetch the RSS file in XML format (this causes a higher bandwidth consumption compared to json format).

If anyone is interested, I can upload the code ...

sriiniivas commented 11 years ago

@AlessandroPappalardo will be appreciated, if you do so bro !

AlessandroPappalardo commented 11 years ago

1) add SMXML (is a very handy lightweight XML parser for iOS) https://github.com/nfarina/xmldocument:

-SMXMLDocument.h
-SMXMLDocument.m

2) in ReviewDownloadManager.h at the beginning, add:

#import "SMXMLDocument.h"

3) change "start" method in ReviewDownloadManager.m to:

- (void)start
{
   backgroundTaskID = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:nil];
   NSString *productID = _product.productID;
   NSString *URLString = [NSString stringWithFormat:@"https://itunes.apple.com/%@/rss/customerreviews/id=%@/sortBy=mostRecent/xml", country, productID];
   NSURL *URL = [NSURL URLWithString:URLString];
   NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:URL];
   self.downloadConnection = [NSURLConnection connectionWithRequest:request delegate:self];
}

4) change "connectionDidFinishLoading" method in ReviewDownloadManager.m to:

 - (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
NSError *error;
SMXMLDocument *document = [SMXMLDocument documentWithData:data error:&error];
// check for errors
if (error) {
    NSLog(@"Error while parsing the document: %@", error);
    return;
}
SMXMLElement *feed = document.root;
//NSLog(@"feed = %@", [feed name]);
//SMXMLElement * firstEntryChild = [feed childNamed:@"entry"];
//NSArray *children = [firstEntryChild children];
// for (SMXMLElement *child in children){
//     NSLog(@"child name = %@", [child name]);
// }
if (1 > 0) // always true
{
    NSString * stringBlank = @"";
    dispatch_async(dispatch_get_global_queue(0, 0), ^ {

        NSMutableArray *reviewInfos = [NSMutableArray new];
        for (SMXMLElement *entry in [feed childrenNamed:@"entry"])
        {

            if ([entry valueWithPath:@"artist"] == nil || [entry valueWithPath:@"artist"]  == stringBlank ){

                //NSArray *children2 = [entry children];
                //for (SMXMLElement *child2 in children2){
                //    NSLog(@"child name = %@", [child2 name]);
                //}
                //NSLog(@"------------");
                //NSLog(@"title = %@", [entry valueWithPath:@"title"]);
                //NSLog(@"updated = %@", [entry valueWithPath:@"updated"]);
                //NSString *updatedReduced = [[entry valueWithPath:@"updated"] substringToIndex:10];
                //NSLog(@"updated Reduced = %@", updatedReduced);
                //NSLog(@"author.name = %@", [entry valueWithPath:@"author.name"]);
                //NSLog(@"content = %@", [entry valueWithPath:@"content"]);
                //NSLog(@"version = %@", [entry valueWithPath:@"version"]);
                //NSLog(@"rating = %@", [entry valueWithPath:@"rating"]);
                //NSLog(@"------------");

                NSDictionary *reviewInfo = [NSDictionary dictionaryWithObjectsAndKeys:
                                            [entry valueWithPath:@"title"],kReviewInfoTitle,
                                            [[entry valueWithPath:@"updated"] substringToIndex:10], kReviewInfoDateString,
                                            [entry valueWithPath:@"author.name"], kReviewInfoUser,
                                            [entry valueWithPath:@"version"], kReviewInfoVersion,
                                            [entry valueWithPath:@"content"], kReviewInfoText,
                                            [NSNumber numberWithInt:[[entry valueWithPath:@"rating"] intValue]], kReviewInfoRating,
                                            nil];
                [reviewInfos addObject:reviewInfo];
            }
        }
        NSManagedObjectContext *moc = [[[NSManagedObjectContext alloc] init] autorelease];
        [moc setPersistentStoreCoordinator:psc];
        [moc setMergePolicy:NSMergeByPropertyObjectTrumpMergePolicy];
        Product *product = (Product *)[moc objectWithID:productObjectID];

        NSSet *downloadedUsers = [NSSet setWithArray:[reviewInfos valueForKey:kReviewInfoUser]];

        //Fetch existing reviews, based on username and country:
        NSFetchRequest *existingReviewsFetchRequest = [[[NSFetchRequest alloc] init] autorelease];
        [existingReviewsFetchRequest setEntity:[NSEntityDescription entityForName:@"Review" inManagedObjectContext:moc]];
        [existingReviewsFetchRequest setPredicate:[NSPredicate predicateWithFormat:@"product == %@ AND countryCode == %@ AND user IN %@", product, country, downloadedUsers]];
        NSArray *existingReviews = [moc executeFetchRequest:existingReviewsFetchRequest error:NULL];
        NSMutableDictionary *existingReviewsByUser = [NSMutableDictionary dictionary];
        for (Review *existingReview in existingReviews) {
            [existingReviewsByUser setObject:existingReview forKey:existingReview.user];
        }

        BOOL changesMade = NO;
        for (NSDictionary *reviewInfo in [reviewInfos reverseObjectEnumerator]) {
            Review *existingReview = [existingReviewsByUser objectForKey:[reviewInfo objectForKey:kReviewInfoUser]];

            // CREATE REVIEW DATE BY COMPONENTS
            NSInteger year = [[[reviewInfo objectForKey:kReviewInfoDateString] substringToIndex:4] intValue];
            NSString * temp1 = [[reviewInfo objectForKey:kReviewInfoDateString] substringFromIndex:5];
            NSInteger month = [[temp1 substringToIndex:2] intValue];
            NSInteger day = [[temp1 substringFromIndex:3] intValue];

            // Combine date and time into components
            NSCalendar *gregorianCalendar = [[NSCalendar alloc] initWithCalendarIdentifier:NSGregorianCalendar];
            NSDateComponents *components = [[NSDateComponents alloc] init];
            [components setYear:year];
            [components setMonth:month];
            [components setDay:day];
            [components setHour:12];
            [components setMinute:0];
            [components setSecond:0];
            // Generate a new NSDate from components.
            NSDate * combinedReviewDate = [gregorianCalendar dateFromComponents:components];

            if (!existingReview) {
                Review *newReview = [NSEntityDescription insertNewObjectForEntityForName:@"Review" inManagedObjectContext:moc];
                newReview.user = [reviewInfo objectForKey:kReviewInfoUser];
                newReview.title = [reviewInfo objectForKey:kReviewInfoTitle];
                newReview.text = [reviewInfo objectForKey:kReviewInfoText];
                newReview.rating = [reviewInfo objectForKey:kReviewInfoRating];
                newReview.downloadDate = [NSDate date];
                newReview.productVersion = [reviewInfo objectForKey:kReviewInfoVersion];
                newReview.product = product;
                newReview.countryCode = country;
                newReview.unread = [NSNumber numberWithBool:YES];
                newReview.reviewDate = combinedReviewDate;

                //NSLog(@"------------");
                //NSLog(@"newReview.title = %@", newReview.title);
                //NSLog(@"newReview.downloadDate = %s", [[newReview.downloadDate description] UTF8String]);
                //NSLog(@"newReview.reviewDate  = %s", [[newReview.reviewDate description] UTF8String]);
                //NSLog(@"newReview.user = %@", newReview.user);
                //NSLog(@"newReview.text = %@", newReview.text);
                //NSLog(@"newReview.productVersion = %@", newReview.productVersion);
                //NSLog(@"newReview.rating = %@", newReview.rating);
                //NSLog(@"------------");

                [existingReviewsByUser setObject:newReview forKey:newReview.user];
                changesMade = YES;
            } else {
                NSString *existingText = existingReview.text;
                NSString *existingTitle = existingReview.title;
                NSNumber *existingRating = existingReview.rating;
                NSString *newText = [reviewInfo objectForKey:kReviewInfoText];
                NSString *newTitle = [reviewInfo objectForKey:kReviewInfoTitle];
                NSNumber *newRating = [reviewInfo objectForKey:kReviewInfoRating];

                if (![existingText isEqualToString:newText] || ![existingTitle isEqualToString:newTitle] || ![existingRating isEqualToNumber:newRating]) {
                    existingReview.text = newText;
                    existingReview.title = newTitle;
                    existingReview.rating = newRating;
                    existingReview.downloadDate = [NSDate date];
                    existingReview.reviewDate = combinedReviewDate;
                    changesMade = YES;
                }
            }
        }

        [psc lock];
        NSError *saveError = nil;
        [moc save:&saveError];
        if (saveError) {
            NSLog(@"Could not save context: %@", saveError);
        }
        [psc unlock];

        if (changesMade && [reviewInfos count] >= 20) {
            dispatch_async(dispatch_get_main_queue(), ^ {
                page = page + 1;
                [self start];
            });
        } else {
            dispatch_async(dispatch_get_main_queue(), ^ {
                if (!canceled) {
                    [self.delegate reviewDownloadDidFinish:self];
                }
            });
        }
    });
} else {
    if (!canceled) {
        [self.delegate reviewDownloadDidFinish:self];
    }
    if (backgroundTaskID != UIBackgroundTaskInvalid) {
        [[UIApplication sharedApplication] endBackgroundTask:backgroundTaskID];
    }
}
}

Consideration: I think, due to the tests I've run, that everything is very stable. @omz could consider to include this code in the master.

Ciao

weakfl commented 11 years ago

@omz would be nice if you could integrate this into the main branch to make reviews available again.

NyKaroly commented 11 years ago

@omz I couldn't agree more with weakfl - "Reviews" were the second most frequently used features for me, and the RSS-based solution above works well. Please merge the changes and bring back the reviews feature. Thanks in advance!

ckoehler commented 11 years ago

I third that. :) I used this app mostly for reviews.

pupasani commented 11 years ago

Yes please get this feature back in. Using fork: https://github.com/ddaddy/AppSales-Mobile just for this feature. But have to manually add other fixes.

ddaddy commented 11 years ago

I have updated my fork to now include this and 2 other fixes. Thanks @abuharsky The only thing not working now in my fork is the promo codes.