SECURING ANDROID SQLITE DB WITH SQLCIPHER AND JNI

Hello Guys,

I am going to tell how to secure android sqlite database, for this we have used :

1) SQLCipher
2) JNI

As you can find simple example (without JNI) in references at last of blog.

1) SQLCipher :
SQLCipher is an open source extension to SQLite that provides transparent 256-bit AES encryption of database files.

Its provides support for Java, Python, Android, iOS, etc.

2) JNI (Java Native Interface):
As to secure it uses 256-bit AES algorithm which requires a key to encrypt or decrypt files.
So we have used JNI to store key securely.

How to integrate in project ?

Steps :

1. Create New Project
2. Give package name
3. Select option “Include c++ support”
4. Create activity and finish.
5.  Add the following line in build.gradle (app)
   compile ‘net.zetetic:android-database-sqlcipher:3.5.4’
6.  Sync the project.

Now good thing is that we do not have to create C++ files and other stuff for JNI.
It will create few files for fetching string values and that much is enough for us.

Calling JNI method from Main Activity

String in JNI

As in case of database, either user can create a new DB or can use existing DB.
So we will see with both cases, these are :
1) Create a new DB , Encrypt and Read & Write DB.
2) Use Existing DB, encrypt it into new DB and Read & Write.

 

1) Create a new DB , Encrypt and Read & Write DB.

In this example we have taken one MainActivity which will load Sqlcipher Library and insert a new record in database and again displaying that record in activity only.
We have also used the Data Binding, to learn about Data Binding please refer :
Data Binding (MVVM) I/O 2016

 

MainActivity.java

package com.example.sqlcipherjniexample;

import android.content.ContentValues;
import android.databinding.DataBindingUtil;
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;

import com.example.sqlcipherjniexample.databinding.MainActivityBinding;

import net.sqlcipher.Cursor;
import net.sqlcipher.database.SQLiteDatabase;

public class MainActivity extends AppCompatActivity {

    // Used to load the 'native-lib' library on application startup.
    // JNI part!
    static {
        System.loadLibrary("native-lib");
    }

    private MainActivityBinding mainActivityBinding;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        mainActivityBinding = DataBindingUtil.setContentView(this, R.layout.activity_main);

        //Loading Sqlcipher Library
        SQLiteDatabase.loadLibs(this);

        //Inserting data in Sqlcipher Sqlite DB!
        insertDataToDb();

        //Reading from DB!
        showDataFromDb();
    }


    private void insertDataToDb() {
        //Note : Wherever we use Sqlite classes, its all from net.sqlite.database.
        SQLiteDatabase db = DatabaseHelper.getInstance(this).getWritableDatabase(stringFromJNI());

        ContentValues values = new ContentValues();
        values.put(DatabaseHelper.WORD, "Word 1");
        values.put(DatabaseHelper.DEFINITION, "First Word");

        db.insert(DatabaseHelper.TABLE_NAME, null, values);

        db.close();
    }


    private void showDataFromDb() {
        SQLiteDatabase db = DatabaseHelper.getInstance(this).getWritableDatabase(stringFromJNI());
        Cursor cursor = db.rawQuery("SELECT * FROM '" + DatabaseHelper.TABLE_NAME + "';", null);
        Log.d(MainActivity.class.getSimpleName(), "Rows count: " + cursor.getCount());

        String dbValues = "";

        if (cursor.moveToFirst()) {
            do {
                dbValues = dbValues + "\n" + cursor.getString(0) + " , " + cursor.getString(1);
            } while (cursor.moveToNext());
        }

        cursor.close();
        db.close();

        mainActivityBinding.sampleText.setText(dbValues);
    }

    /**
     * A native method that is implemented by the 'native-lib' native library,
     * which is packaged with this application.
     */
    public native String stringFromJNI();
}

 

activity_main.xml (in layout folder)

<layout xmlns:android="http://schemas.android.com/apk/res/android">

    <data class="MainActivityBinding" />

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent">

        <TextView
            android:id="@+id/sample_text"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content" />

    </RelativeLayout>
</layout>

 

 

Databasehelper.java

package com.example.sqlcipherjniexample;

import android.content.Context;

import net.sqlcipher.database.SQLiteDatabase;
import net.sqlcipher.database.SQLiteOpenHelper;

/**
 * Created by Sumeet on 09-04-2017.
 */

public class DatabaseHelper extends SQLiteOpenHelper {

    private static DatabaseHelper instance;

    public static final int DATABASE_VERSION = 1;
    public static final String DATABASE_NAME = "DICTIONARY_NEW";

    public static final String TABLE_NAME = "FTS";
    public static final String WORD = "word";
    public static final String DEFINITION = "definition";

    private static final String TEXT_TYPE = " TEXT";
    private static final String SQL_CREATE_ENTRIES = "CREATE TABLE " + TABLE_NAME + " (" + WORD + TEXT_TYPE + "," + DEFINITION + TEXT_TYPE + " )";


    private static final String SQL_DELETE_ENTRIES = "DROP TABLE IF EXISTS " + TABLE_NAME;

    public DatabaseHelper(Context context) {
        super(context, DATABASE_NAME, null, DATABASE_VERSION);
    }

    static public synchronized DatabaseHelper getInstance(Context context) {
        if (instance == null) {
            instance = new DatabaseHelper(context);
        }
        return instance;
    }

   
    @Override
    public void onCreate(SQLiteDatabase sqLiteDatabase) {
        sqLiteDatabase.execSQL(SQL_CREATE_ENTRIES);
    }

    @Override
    public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) {
        sqLiteDatabase.execSQL(SQL_DELETE_ENTRIES);
        onCreate(sqLiteDatabase);
    }
}

 

build.gradle (Module : app)

apply plugin: 'com.android.application'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.1"
    defaultConfig {
        applicationId "com.example.sqlcipherjniexample"
        minSdkVersion 15
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                cppFlags ""
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }

    //Its for data binding.
    dataBinding {
        enabled = true
    }

}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:25.3.1'
    compile 'com.android.support.constraint:constraint-layout:1.0.2'
    testCompile 'junit:junit:4.12'
    compile 'net.zetetic:android-database-sqlcipher:3.5.4'
}

 

And then we finally test it with Virtual Device / Emulator, with configurations as :
Device : Nexus 5
API level : 23
CPU/ABI : x86

And it worked successfully! 🙂

Please check screen shot below :

 

The most important part is when I tested it with the same example with different device and its configuration are :
Device : Nexus 5
Api level : 25
CPU/ABI : X86_64

So app got crashed!
Please check the screen shot of both the device :

So what is the problem and I started searching about it.
Then I found that due this :
SqlCipher does not support 64 bit CPU Architecture

And they have mentioned in their reply :

While there is a pending pull request to add both arm64v8a and x86_64 support to SQLCipher for Android, we are still awaiting integration [2] from OpenSSL directly on these specific platforms before we can continue with the merge. We are unaware of the priority the OpenSSL team has on adding direct support for those platforms within their build process. As soon as that becomes available we will revisit adding support to SQLCipher for Android.

That said, as @commonsguy mentioned, the 32 bit binaries will continue to work on a 64 bit device, however you can not include any other 64 bit binaries within the application. We look forward to expanding further platform support for SQLCipher for Android, however we need OpenSSL to officially support those platforms first.

And when I went in more detail by analyzing our debug.apk.
Android Studio -> Build -> Analyze Apk -> select apk (app-debug.apk in our case)

I found .so files for all types of CPU architecture (32 and 64 bits both):

Detail view :

 

But again here we can notice that Sqlcipher generated any .so file not supported architectures like X86_64, arm64-v8a, etc.

So why is it still giving me crash (Still in a doubt !).

Then finally with more searching and struggles I found that :

Sqlcipher for Android Native Libs (JNI)

Just check for the comments and here in more detail they have mentioned :

Due to Android’s preferential loading process, if, for example, your APK is including other third-party libraries that contain native libraries that target architectures other than armeabi, armeabi-v7a, or x86, Android will attempt to load all native libraries from within the folder that is the closest match for its platform architecture, thus causing the error above when it attempts to locate SQLCipher for Android within a 64 bit platform folder.

And now solution to it is :

This issue can be addressed by removing any included native libraries for architectures other than armeabi, armeabi-v7a, and x86; this allows the Android device to load the 32 bit version of all libraries, even on a 64 bit device. For gradle users, you can utilize the ABI split feature.

 

As per there above suggestion we should APK-Split, which is nothing but to make apk based on denisty (hdpi, xxhdpi, etc.) or for different CPU architecture (x86, mips, etc.)

But here come an another problem to maintain Multiple Apks, which is much complex to maintain and hectic process for developers also.
So we found with some other solution, that is to have only One Apk and which supports all 32 bits ABI/CPU (especially to support SqlCipher) which we manually specify.

Making One Apk with multiple native support (Please see its “Specify ABIs” section.)

And we made following changes in our build.gradle (module:app).
Added ndk block with abiFilters (this will create a single apk with the ABIs we will specify)

 

defaultConfig {
    ...//Your code
    ndk {
        abiFilters 'x86', 'armeabi', 'armeabi-v7a'
    }
}

 

And finally it worked ! 🙂

Please see the following screen shot :

 

And when we again analyze apk :

 

You will see that the ABIs we manually defined it created .so files for that only.
As android allows all 32 bit library to work with 64 bit so its working fine with our requirement and this will reduce size also.

Now lets try to extract DB and test it whether its encrypted or not.
Android Studio -> Tools ->Android -> Android Device Monitor -> then choose device -> data ->choose app package id (in our case its “com.example.sqlcipherjniexample”) -> databases -> Database Name (“Dictionary_new” in our case).

Now If user tries to open it then it will get an error “file is encrypted or is not a database”

So finally, we got complete working example Sqlcipher with JNI.

Please find the working example at  my github account :
SqlCipher with JNI Example

Now lets see for if an app has existing database and encrypt it and read & write operation.

2) Use Existing DB, encrypt it into new DB and Read & Write.

For this we have taken a predefined database named as DICTIONARY_NEW which is having 5 records.

 

Now here we have copied this existing database in assets folder.

 

Make sure this Database is  not encrypted.

MainActivity.java

package com.example.sqlcipherjniexistsdb;

import android.databinding.DataBindingUtil;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.Log;
import android.widget.TextView;

import com.example.sqlcipherjniexistsdb.databinding.MainActivityBinding;

import net.sqlcipher.Cursor;
import net.sqlcipher.database.SQLiteDatabase;

import java.io.IOException;

public class MainActivity extends AppCompatActivity {

    // Used to load the 'native-lib' library on application startup.
    static {
        System.loadLibrary("native-lib");
    }

    private MainActivityBinding mainActivityBinding;
    private Databasehelper databaseManager;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        mainActivityBinding = DataBindingUtil.setContentView(this, R.layout.activity_main);


        //Loading Sqlcipher Library
        SQLiteDatabase.loadLibs(this);

        try {
            //Loading data from existing db to encrypted DB!
            loadAllData();

            //Reading from DB!
            showDataFromDb();
        } catch (IOException e) {
            e.printStackTrace();
        }

    }


    public void loadAllData() throws IOException {
        try {
            databaseManager = Databasehelper.getInstance(MainActivity.this);
            databaseManager.createOpenDataBase(stringFromJNI());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }


    private void showDataFromDb() throws IOException {
        SQLiteDatabase db = Databasehelper.getInstance(this).getWritableDatabase(stringFromJNI());
        Cursor cursor = db.rawQuery("SELECT * FROM '" + Databasehelper.TB_FTS + "';", null);
        //Log.d(MainActivity.class.getSimpleName(), "Rows count: " + cursor.getCount());

        String dbValues = "";

        if (cursor.moveToFirst()) {
            do {
                dbValues = dbValues + "\n" + cursor.getString(0) + " , " + cursor.getString(1);
            } while (cursor.moveToNext());
        }

        cursor.close();
        db.close();

        mainActivityBinding.sampleText.setText(dbValues);
        databaseManager.close();
    }


    /**
     * A native method that is implemented by the 'native-lib' native library,
     * which is packaged with this application.
     */
    public native String stringFromJNI();
}

 

activity_main.xml

<layout xmlns:android="http://schemas.android.com/apk/res/android">

    <data class="MainActivityBinding" />

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent">

        <TextView
            android:id="@+id/sample_text"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content" />

    </RelativeLayout>
</layout>

 

 

Databasehelper.java

package com.example.sqlcipherjniexistsdb;

import android.content.Context;
import android.util.Log;

import net.sqlcipher.database.SQLiteDatabase;
import net.sqlcipher.database.SQLiteOpenHelper;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.sql.SQLException;

/**
 * Created by Sumeet on 23-12-2016.
 */

public class Databasehelper extends SQLiteOpenHelper {
    public static String DB_PATH;
    public static String DB_NAME = "DICTIONARY_NEW";
    //Version Done (2) on 30-04-17
    public static final int DB_VERSION = 1;
    public static final String TB_FTS = "FTS";
    private net.sqlcipher.database.SQLiteDatabase myDB;
    private Context context;
    private static Databasehelper instance;
    public static String KEY;


    public static synchronized Databasehelper getInstance(Context context) {
        if (instance == null) {
            DB_PATH = BuildConfig.DB_PATH;
            instance = new Databasehelper(context);
        }
        return instance;
    }

    public Databasehelper(Context context) {
        super(context, DB_NAME, null, DB_VERSION);
        this.context = context;
    }

    @Override
    public void onCreate(net.sqlcipher.database.SQLiteDatabase sqLiteDatabase) {

    }

    @Override
    public void onUpgrade(net.sqlcipher.database.SQLiteDatabase sqLiteDatabase, int oldVersion, int newVersion) {
    }


    /***
     * Check if the database doesn't exist on device, create new one
     *
     * @throws IOException
     */
    public void createOpenDataBase(String encryptKey) throws IOException {
        //VIP -> checkDataBase method, return false if press Apps in background button (Bottom Right button in device) and remove it from back-stack.
        boolean dbExist = checkDataBase();
        //This key is used to encrypt and decrypt the db!
        KEY = encryptKey;
        if (!dbExist) {
            try {
                //we should pass empty string, when we have to open any other database which is not encrypted by SqlCipher!
                this.getReadableDatabase("");

                //When we use encrypted DB (so that no one can get the data after extracting apk->assets and -> DB)
                //So we are keeping encrypting file only.
                //We made the DB using the above KEY!
                //this.getReadableDatabase(KEY);

                //first we copy from assets folder to app's internal storage!
                copyDataBase();
                //Then we make another db and copy the data in encrypted manner!
                encryptDB();
            } catch (IOException e) {
                Log.d("createOpenDatabase", e.getMessage());
            } catch (Exception e) {
                Log.d("createOpenDB", e.getMessage());
            }
        }

        //Then opening DB!, This is not required as we can db gets open when we call getReadable/getWritable!
        try {
            openDataBase();
        } catch (net.sqlcipher.SQLException e) {
            e.printStackTrace();
        }
    }

    /***
     * Check if the database is exist on device or not
     * This method will throw an error when it does not get any database.
     *
     * @return
     */
    private boolean checkDataBase() {
        net.sqlcipher.database.SQLiteDatabase tempDB = null;
        try {
            String myPath = DB_PATH + DB_NAME;
            tempDB = net.sqlcipher.database.SQLiteDatabase.openDatabase(myPath, KEY, null, net.sqlcipher.database.SQLiteDatabase.OPEN_READWRITE);
        } catch (net.sqlcipher.database.SQLiteException e) {
            Log.d("checkDataBase", e.getMessage());
        }
        if (tempDB != null)
            tempDB.close();
        return tempDB != null ? true : false;
    }


    /***
     * Copy database from source code assets to device
     *
     * @throws IOException
     */
    public void copyDataBase() throws IOException {
        try {
            InputStream myInput = context.getAssets().open(DB_NAME);
            String outputFileName = DB_PATH + DB_NAME;
            OutputStream myOutput = new FileOutputStream(outputFileName);

            byte[] buffer = new byte[1024];
            int length;

            while ((length = myInput.read(buffer)) > 0) {
                myOutput.write(buffer, 0, length);
            }

            myOutput.flush();
            myOutput.close();
            myInput.close();
        } catch (Exception e) {
            Log.d("copyDatabase", e.getMessage());
        }

    }


    /***
     * Open database
     *
     * @throws SQLException
     */
    public void openDataBase() throws net.sqlcipher.SQLException {
        String myPath = DB_PATH + DB_NAME;
        myDB = net.sqlcipher.database.SQLiteDatabase.openDatabase(myPath, KEY, null, net.sqlcipher.database.SQLiteDatabase.OPEN_READWRITE);
    }


    public SQLiteDatabase getMYDB() {
        return myDB;
    }

    private void encryptDB() throws IOException {
        String myPath = DB_PATH + DB_NAME;
        File originalFile = context.getDatabasePath(myPath);

        File newFile = File.createTempFile("sqlcipherutils", "tmp", context.getCacheDir());

        //As this was for default db (without encryption)!
        net.sqlcipher.database.SQLiteDatabase existing_db = net.sqlcipher.database.SQLiteDatabase.openDatabase(myPath, "", null, net.sqlcipher.database.SQLiteDatabase.OPEN_READWRITE);

        //And now we are using already encrypted DB!
        //KEY
        //net.sqlcipher.database.SQLiteDatabase existing_db = net.sqlcipher.database.SQLiteDatabase.openDatabase(myPath, KEY, null, net.sqlcipher.database.SQLiteDatabase.OPEN_READWRITE);
        String newPath = newFile.getPath();
        existing_db.rawExecSQL("ATTACH DATABASE '" + newPath + "' AS encrypted KEY '" + KEY + "';");
        existing_db.rawExecSQL("SELECT sqlcipher_export('encrypted');");
        existing_db.rawExecSQL("DETACH DATABASE encrypted;");

        existing_db.close();
        //Deleting original plain text db!
        originalFile.delete();
        //Renaming the new db same as old db!
        newFile.renameTo(originalFile);

    }


    @Override
    public synchronized void close() {
        if (myDB != null) {
            myDB.close();
        }
        super.close();
    }

}

 

build.gradle (Module:app)

apply plugin: 'com.android.application'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.1"
    defaultConfig {
        applicationId "com.example.sqlcipherjniexistsdb"
        minSdkVersion 15
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                cppFlags ""
            }
        }

        ndk {
            abiFilters 'x86', 'armeabi', 'armeabi-v7a'
        }

    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
            buildConfigField "String", "DB_PATH", "\"/data/data/com.example.sqlcipherjniexistsdb/databases/\""
        }
        debug{
            buildConfigField "String", "DB_PATH", "\"/data/data/com.example.sqlcipherjniexistsdb/databases/\""
        }
    }
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }

    //Its for data binding.
    dataBinding {
        enabled = true
    }

}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:25.3.1'
    compile 'com.android.support.constraint:constraint-layout:1.0.2'
    compile 'net.zetetic:android-database-sqlcipher:3.5.4'
    testCompile 'junit:junit:4.12'
}

 

And its finally working !

 

Now if user extracts DB also from App, then also he will get encrypted database.
As we have deleted the original (not encrypted) database from device and replaced it by encrypted one.

 

 

 

Please find its working example at my github account :
SqlCipher with JNI for Existing DB
Kindly let me know for any concern.

Links :
1) https://github.com/sqlcipher/android-database-sqlcipher
2) https://www.zetetic.net/sqlcipher/sqlcipher-for-android/
3) http://lomza.totem-soft.com/tutorial-add-sqlcipher-to-your-android-app/
4) https://discuss.zetetic.net/t/sqlcipher-for-android-native-library-support/1440
5) https://developer.android.com/studio/projects/add-native-code.html