Remote debugging an embedded WebView in Android

And there I was, thinking that I was done with AzureStorageExplorer for Android v1.0.0. But, nope. In my last round of testing, I couldn’t even login using my work account. The Sign In button didn’t do anything. I went back a few versions to see if the Android update for WebView could have done something to break it. That wasn’t it. The OAuth client library I was using had the recommended code for enabling Javascript in the WebView client so I knew that wasn’t it. I was able to login through the Azure OAuth flow using my personal Azure account. So, I knew there was something about the enterprise account login screen, which could have changed recently that ended up breaking the authentication in the app. It turns out that it did. Read on to know how I found out.

The first step was to prove that something was indeed broken in the login screen for enterprise accounts in Azure in embedded WebViews. To do this, I looked up Chrome’s remote debugging options and found this page describing how to remotely debug WebViews in mobile apps. So, I enabled the setting as recommended by Google, which required adding this piece of code in my activity’s onCreate(Bundle) method.


if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
WebView.setWebContentsDebuggingEnabled(true);
}

I ran the app, launched Chrome’s Developer Tools, and connected to the app’s WebView. This is what I found..

2016-12-26_09-49-21

That last error in the console is what was logged when I pressed the Sign In button after entering my work account credentials for Azure. It turns out that the page’s JS now has code to store something in the browser’s localStorage. The reason this isn’t a problem, if you were logging into Azure through a desktop or even a mobile browser, is because localStorage is enabled for every site by default in the non-embedded browser view. But for a WebView in an Android app you need to explicitly enable the DOM storage, just like you need to enable JS through the WebSettings class.

After finding this, it was easy to know where I needed to make the fix. So, I set out to fork the source repo of the OAuth library I was using, made the necessary changes and created a pull request. If you are using the same library and have run into this issue, you could clone my repo and build the project to produce the patched .jar (or an .aar), which you can use in your project directly until the author of the library can get to my pull request (if at all!).

Advertisements
Image

Unofficial Tesla Android client for controlling your Model S

Tesla Motors

Tesla Motors

GitHub link: https://github.com/praneetloke/MyTesla

But there’s already an official app from Tesla for the iOS and Android, then why this? Because I wanted to and besides there aren’t any open source Android clients. There are couple of Ruby clients and a node.js client. The node.js client is particularly of interest to me since it has visual examples of each API it supports. The telemetry streaming API is my favorite. If you remember, that’s what Tesla used to debunk the NY Times reporter for his fake report on the Model S sometime ago.

Anyway, back to the MyTesla client. You can fork it, download it, modify it, do whatever you want with it. This client is unstable, unofficial, and most importantly, unverified (since I don’t own a Tesla Model S, unfortunately). So please use this with caution. If you find bugs, please raise an issue in my repo or you can fork-fix-pull. If you are able to help with its stabilization by testing this on your Tesla Model S, please let me know. You can hit me up on G+ or LinkedIn.

There are 3 REST API clients for your use in this project. LoginClient, VehicleStatusClient and VehicleCommandClient. I have already pulled in these 3 REST interfaces into a custom Android Application class called TeslaApplication. I did this to remove any dependency from consumers having to pull in AndroidAnnotations as well. On a side note, you should check out AndroidAnnotations. It is awesome! Anyway, in your project, you can simply extend the TeslaApplication and be on your way. You don’t need to put anything in it. I have not actually tested to see if such an extension is actually required if you were to reference this project but be my guest.

I have also made a LoginActivity which has a boilerplate login form which you can present to your users. It handles submitting the email and password to the API and inspecting the response to see if it was successful. I actually plan to change this to a login dialog instead or perhaps have both since a dialog only needs a layout. Then you can choose either depending on your needs. When login fails, it currently doesn’t do anything. I am yet to work on that. I also need to work on some cookie transfer from LoginClient to the other two clients because from what I saw in the AndroidAnnotations sources, there doesn’t seem to be a unified storage for cookies acquired by REST interfaces.

When you look at the library I made, you will notice that I didn’t use primitives. That’s because of the nature of the API itself. It’s unofficial and there are unconfirmed properties whose values are unknown and sometimes null. So this being Java, I couldn’t use primitives in some cases. For those that had confirmed values I could have used primitives but I felt I needed to be consistent rather than have you guess what you’ll be using when you inspect an object. And yes, I am talking about primitives and objects because I actually went ahead and created model classes (POJOs) for all of the endpoints. This should make interaction way easier. It uses Gson for type conversion. I chose Gson over Jackson mapper for its lightweight and performance. Gson doesn’t have all of the features Jackson has but it does the job, fast too.

If you have watched this clip of the guy issuing commands to a Tesla Model S, you’ll be at least half as excited as I was to find a REST client and play with it if you have a Tesla Model S. Of course, you would more likely already have the official Tesla app. But if you are into programming and diving into things on your own, this is for you. I wish my VW CC was capable of something like this.

Credits

  • Tim Dorr (and many others that commented on each API endpoint in the Apiary blueprint) for his excellent Apiary documentation based on his findings. He has implemented his very own ruby implementation of the API here.
  • AndroidAnnotations
  • Spring Android

Other clients

  • node.js
  • Ruby. For Ruby, there’s also the one from Tim Dorr himself.

My struggles with Android’s CalendarContract introduced in ICS

While working on some upgrades for Squiggly I wanted to subscribe to Android’s calendar provider events. This is how you would do it.

Assuming you have a receiver in your app, if you have declared it in the manifest xml, then add this to get provider_changed intents. Thanks to this stackoverflow question.

<intent-filter>
    <action android:name="android.intent.action.PROVIDER_CHANGED"/>
    <data android:scheme="content"/>
    <data android:host="com.android.calendar"/>
</intent-filter>

Now, the above intent-filter is missing a few things that could have made my interaction with the calendar provider easier. They are the attributes paths, pathPattern and pathPrefix for the “ part of the intent-filter. Unfortunately, even specifying those attributes won’t do you any good because the CalendarProvider class itself doesn’t specify those parameters on the intent that is fired from within.

The other problem is that the intent that is fired from CalendarProvider doesn’t even carry the event id that is passed to it from the methods that call it. See sendUpdateNotification from CalendarProvider2.java in the JellyBean source code here. This method sends a message to its Handler instance which then broadcasts the intent for provider_changed action. ref: this line. The problem is this simply obtains a message from the message pool and passes the value that sets the what of the Message instance returned. Instead, they could have done this:

//get a message using the obtainMessage(int what, Object obj) version
Message msg = mBroadcastHandler.obtainMessage(UPDATE_BROADCAST_MSG, new Long(eventId));

With the eventId now getting lost with the aforementioned line#4478 from CalendarProvider2.java, the handler now doesn’t even add any extras to the intent before broadcasting it. The control eventually reaches doSendUpdateNotification, which as you can see does nothing but set the action in the intent and broadcasts it.

What’s even more frustrating is that if you look at the usages of this method, the callers actually do pass the eventId and the actions that trigger a call to this method are actually the ones I want to know the event ID for!!

Usages for sendUpdateNotification in JellyBean r1

Usages for sendUpdateNotification in JellyBean r1

Well, looks like the latest KitKat revision still hasn’t changed this. Until then, I guess I will have to scan the instances table whenever the provider_changed action is received. 😦

If you care about this issue and want to bring this to the Android team’s attention then please vote for this issue.

Dynamic entries in Android’s MultiSelectListPreference

So I was finally excited that my brother and I decided to drop support for Gingerbread for the next iteration of Squiggly and that allows me to do things like use the MultiSelectListPreference and use the local Calendar API exclusively.

But there is a drawback with MultiSelectListPreference. You have to pre-define the string arrays for entries and entryValues. This sucks. This is why I have now added CustomMultiSelectListPreference to CustomPreferences. I have followed the same pattern which I did with CustomListPreference by implementing the IDynamicProvider which I introduced a while back when I needed to populate a regular list preference dynamically.

This is the class to reference in your preference xml file.

To use it:

<com.myappfactory.preferences.CustomMultiSelectListPreference
            android:key="@string/calendarsToMonitorKey"
            android:dependency="@string/enableCalendarBackgroundServicesKey"
            android:title="@string/calendarsToMonitorTitle"
            android:dialogTitle="@string/calendarsToMonitorDialogTitle"
            android:summary="@string/calendarsToMonitorSummary"
            customPreference:dynamicEntriesProvider="your.app.package.ClassThatImplementsIDynamicProvider"
            customPreference:dynamicEntryValuesProvider="your.app.package.ClassThatImplementsIDynamicProvider"
            customPreference:selectAllValuesByDefault="true"
            android:entries="@array/empty_array"
            android:entryValues="@array/empty_array"
            android:defaultValue="@array/empty_array"/>

selectAllValuesByDefault does exactly what it says. It selects all the values that are dynamically added. The class that provides these dynamic entries and entryValues need to implement IDynamicProvider. The no-argument version of populate method will no longer be supported. So be sure to implement the populate that accepts a Context. CustomMultiSelectListPreference first calls populate then calls getItem. I have tried to keep the same naming convention that Android’s adapter classes use. So you will see getCount and getItems in there.

Here’s a preview of what it will look like:

CustomMultiSelectListPreference

CustomMultiSelectListPreference

Ford TDK

Guess what?? Last week, I got the Ford TDK which I won from participating in an app idea contest held by Ford. Here’re the pictures!

TDK power cord and instructions manual.

Instructions and power cord for the Ford/Lincoln TDK 3.0

Ford/Lincoln TDK

Ford/Lincoln TDK

[UPDATE] Weird problem with GridView OnScrollListener and list navigation listener

It turns out that the problem was much simpler than I had originally thought.

The Problem

It was actually two fold. The scroll listener for some reason fires the change event as soon as the gridview’s adapter is set even when there are no items in it. My scroll listener was something like this:

private AbsListView.OnScrollListener mOnScrollListener = new AbsListView.OnScrollListener({
  @Override
  public void onScrollStateChanged (AbsListView view, int scrollState) {}

  @Override
  public void onScroll (AbsListView view, int firstVisibleItem, int visibleItemCount, int totalItemCount) {
    if ((firstVisibleItem + lastVisibleItem) >= totalItemCount && !mAsyncTasksPending) {
      loadMore();
    }
  }
});

The above code ended up calling the loadMore(); even before the gridview’s adapter was loaded with the first page results from the service. That ended up making two requests wastefully. The second problem is because of how I had declared the loadMore(); method. I use AndroidAnnotations to reduce boilerplate code. I annotated loadMore(); with @Background to execute the service request in a background thread to prevent the UI from locking up. But, I didn’t need to do this since I was already taking care that the service request be executed in a background thread using another library called async-http, which by the way is freaking awesome! My guess is, when I set the method to be executed in a background thread it was holding up the gridview’s listener thread somehow which caused the events to accumulate in the call stack. This would probably explain why when I paused the activity and resumed it, it would function normally and the moment I scrolled it and caused it to call loadMore() it would freeze again and none of the callbacks would fire.

A little more than the solution

First, what I ended up doing was removing the redundant @Background annotation from loadMore();. Then, I realized that my activity was a little bloated. So, I created a fragment and shoved the gridview, and managing its adapter into the fragment. I was then able to keep my activity lean. By doing so, I have also opened up the possibility of introducing additional fragments to handle different views if necessary in the future which makes this all the more easy to manage without bloating my activity.

Next, I updated the gridview’s onScroll callback. It was still being called as soon as the adapter was set in the gridview. Initially, the adapter is empty and I don’t want it to call loadMore(); when the first page results haven’t been filled in the adapter yet. Loading the first page is handled through the list navigation listener since that sets the currently selected item for which I then go and fetch the list of images. So now, this is how my updated OnScrollListener looks.

private AbsListView.OnScrollListener mOnScrollListener = new AbsListView.OnScrollListener({
  @Override
  public void onScrollStateChanged (AbsListView view, int scrollState) {}

  @Override
  public void onScroll (AbsListView view, int firstVisibleItem, int visibleItemCount, int totalItemCount) {
    //only load more if the visibleItemCount is > 0, and the user has reached the last row of visible items
    if (visibleItemCount > 0 && ((firstVisibleItem + lastVisibleItem) >= totalItemCount) && !mAsyncTasksPending) {
      loadMore();
    }
  }
});

Why I dumped my custom image manipulation code from Transitions

After spending several nights and weekends implementing and testing the code I had written for image resizing, it still didn’t work for some popular devices. This was unacceptable. As I gave this a lot of thought, I realized that I was doing it wrong. No. Not just the code. My effort. As much as the engineer that I am, I wanted to make it work. I was very interested in solving it. I wanted to own it much like every other coding problem I try to solve. But, as an Engineer, I also don’t want to “reinvent the wheel” if I am not going to improve it. I certainly wasn’t improving it.

So, for the upcoming release for Transitions, I trashed all of the code I wrote for image manipulation and simply popped in Square’s Picasso library. Why? Because, it’s simply that much less code I need to change/maintain. It is so much easier to look at my code now with all the complex bitmap decoding stuff gone. I’ll let the experts handle that. And, now I can concentrate what really matters. Putting features together. I am still innovating with Transitions. I added a feature to my open source library CustomPreferences which allows you to load a list preference from the database dynamically. It’s very useful when you let your users build their own database of things and you want to allow them to pick from that list. Currently, you can’t do this with the preference classes out-of-the-box. But, you can, with CustomPreferences.

 

 

Closure function vs defined (named) function

I wrote a performance test on jsperf.com to measure the difference between calling a closure function from within a loop and calling a defined function within another loop for the same number of iterations. See the difference and run the test yourself.

PHP file download

So I have been working on a PHP website for a freelance job and had to write few lines of code to download a file from the database. Very simple right? Yeah that’s what I thought too. I mean it was < than 20 lines of code. Why wouldn’t it work? What could go wrong? That’s where I was wrong. This is what I had at first:

if(isset($_GET['id']))
{
  // if id is set then get the file with the id from database
  $id = $_GET['id'];
  $application = new Applications();
  $application->get($id);
  header("Content-length: $application->size");
  header("Content-type: $application->type");
  header("Content-Disposition: attachment; filename=$application->name");
  $content = $application->data;
  echo $content;
}
exit;

The problem I experienced was that a file would upload (different piece of code) fine. I knew that because the size attributes match when the file is written to the db. When I actually try to download the file back from the db using the script above, the file gets corrupted. I had no idea why! The weird (but made sense later) part was that plaintext files were fine no matter how large they were. It was binary files like images, DOCs, PDFs etc. So, I scoured the internet searching for answers and finally came across one thread somewhere where there was a similar discussion from over a year ago and someone said that the file stream would get random whitespace characters when downloading the file. Then it hit me, I remember seeing whitespace at the beginning of the stream when I had opened the corrupt file using notepad. The solution was to ob_clean() before you begin writing the headers and the content stream itself. You see one of the important points about writing header information to the client is that there shouldn’t be any whitespace before writing headers.

I modified the above code then to this:

if(isset($_GET['id']))
{
  // if id is set then get the file with the id from database
  $id = $_GET['id'];
  $application = new Applications();
  $application->get($id);
  ob_clean(); //clears output buffer in php..see php manual
  header("Content-length: $application->size");
  header("Content-type: $application->type");
  header("Content-Disposition: attachment; filename=$application->name");
  $content = $application->data;
  echo $content;
}
exit;

It would then download without any problems. At this time I should also caution you about dangling newline/whitespace in the php file itself which handles the download. The reason why plaintext files were ok is because plaintext files are not written in binary so whitespace or not it doesn’t matter for those file types and they will upload/download fine, at least in my case they did. Do you have a better way to handle this? If so, please leave a comment. Let me know about it.

Android NetworkOnMainThreadException

Android has been lenient about network access on the main thread in prior versions and now with ICS, they are cracking down on us.

It is bad practice however to have operations that are intensive and for a network operation it is even more sensitive since the network speed is a varying factor that can block the UI for an unknown amount of time. Although it is annoying to have your app crash all of a sudden in new Android versions because you (and by that I mean, I) made a bad programming mistake, it is for your own good to consider porting your network access code to an AsyncTask. Writing an AsyncTask is shit easy.

private class SomeTask extends AsyncTask<Void, Void, Void>{

@Override

protected Void doInBackground(Void…params){

//your network access code goes here

}

}

and then in your UI thread or main thread, just instantiate this task and execute it and you’re done. Obviously, I have simplified the AsyncTask by not going into much details about the arguments and result codes etc..You can read all about that here. I can’t tell you how much I love the Android documentation. Say what you want about their fragmentation problem; I just love their documentation. Just amazing. There’s nothing you can’t find in there unlike Apple’s. I spent a whole day trying to figure out how to create those provisioning and distribution certificates and there’s not an article on the internet that ties it all together!