\addtocontents{toc}{\protect\vspace{\beforebibskip}} % Place slightly below the rest of the document content in the table

%************************************************
\chapter{GBoard Application and Module}
\label{ch:module}
%************************************************

\begin{justify}
In this chapter, we describe our developed GBoard standalone application called GBoard4A, the integration with Autopsy digital forensics software and the development of two modules for Autopsy.
\end{justify}

\begin{justify}
Although digital analysis tools might be complex to implement, the end-user should not experience complexity when working with the software. The usage of any software tool should be simple, user-friendly, and effective. A software tool conceptualized with this in mind provides to any user a better working experience. It also improves productivity by performing daily work tasks without much trouble, no matter what level of knowledge the user has outside their working environment. Providing this was something we had in mind when planning how to deliver the final product.
\end{justify}

\section{Standalone application}

\begin{justify}
For this project, we decided to develop a standalone \gls{cli} application separated from the Autopsy module. This decision came along with the advantage of future extensibility and reusability of the application with other forensics tools and the philosophy of software isolation to achieve better software quality by performing more isolated testing.
\end{justify}

\begin{justify}
Because of the constant evolution of the software, extensibility and reusability is a key feature to reach a faster software growth and it is part of a good system design principle to effortlessly extend a piece of software to different systems and environments \parencite{DesignExtensibility}. To achieve that we created a simple command-line \gls{api}. With that approach, it is possible to any software integrate our application and build a module, easily. This is particularly useful for our usecase because this work includes the development of two different modules for the Autopsy software.
\end{justify}

\begin{justify}
Software isolation is part of the UNIX philosophy for creating minimalist and modular software and it is very important for the development of quality software. Quoting \textcite{UNIXPhilosophy}, "Write programs that do one thing and do it well". By creating a sub-program only to fetch GBoard information, it is possible to detect errors more easily due to the reduced number of tasks it needs to perform and therefore test the software behaviour in a more effective way. This is important because software tends to become complex and hard to reproduce in a monolithic scenario.
\end{justify}

\subsection{Operating modes}

\begin{justify}
The standalone application has two distinguishing modes for different purposes: root analysis, which performs a full analysis from a given GBoard root path, and multiple path analysis which can perform multiple analyses on different files or folders by giving multiple entries.
\end{justify}

\begin{justify}
Root path analysis is useful when a full dump of GBoard internal folder is done and it is intended to preform a full forensic analysis. This is mostly used with the Autopsy modules to report the desired results.
\end{justify}

\begin{justify}
Multiple path analysis, on the other hand, is useful when a partial dump is available or the GBoard files are not organized in the same way GBoard does internally. This is possible with an auto-detection mechanism that seeks which file is. Additionally, this is particularly useful when the intention is to only analyse a single file.
\end{justify}

\begin{listing}[ht]
\begin{textcode}
Some information about the program

-r --root-dir GBoard root directory analysis (must be used alone)
-d      --dir GBoard directory analysis
-f     --file GBoard file analysis
-t     --type Output format type (default: json)
-o   --output Output file path of analysis report
-v  --verbose Print extra information on the analysis
-h     --help This help information.
\end{textcode}
\caption{Ouput of the help information of the standalone application}
\label{listing:GBoardStandaloneHelpOutput}
\end{listing}

\begin{justify}
Listing \ref{listing:GBoardStandaloneHelpOutput} shows the output of the standalone application when \path{--help} argument is given. To run a root path analysis, the argument \path{--root-dir}/\path{-r} should be present followed by the path of the GBoard internal folder dump. For multiple path analysis, either \path{--file}/\path{-f} or \path{--dir}/\path{-d} should be present followed by the file or directory path to analyse. All the mentioned arguments can be provided multiple times, and root path analysis is only allowed when \path{--root-dir}/\path{-r} is used alone.
\end{justify}

\begin{justify}
To be able to switch between different analysis reports is possible to specify a report type with \path{-t}/\path{--type} and a custom output path instead of standard output, which is the default, with \path{-o}/\path{--output} followed by an output file path. The available report types are \path{json} for \gls{json} format and \path{html} for \gls{html} format. Additionally it is possible to run the application with \path{-v}/\path{--verbose} to print extra information for debugging purpose.
\end{justify}

\subsection{Output Report Types}

\begin{justify}
This application implements two types of reporting, useful for different situations: \gls{json} and \gls{html}.
\end{justify}

\subsubsection{JSON Type}

\begin{justify}
\gls{json} and \gls{json}-like data structures are common and widely used by software industry for bridging two applications or services and still maintain a readable file format. These data structures are often implemented part of a programming language standard library because of its popularity \parencite{JSONWebsite}. The \gls{json} reporter is particularly useful for modularization. It provides a way to easily integrate with other tools by using an easy language to parse. 
\end{justify}

\begin{listing}[ht]
\begin{jsoncode}
{
	"rootPath": "/location/to/gboard-folder",
	"dictionaries": [
		{
			"path": "./gboard-folder/databases/PersonalDictionary.db",
			"entries": []
		}
	],
	"trainingcache": [
	]
}
\end{jsoncode}
\caption{Reduced example of the \acrshort{json} output report}
\label{listing:OutputJSONStandaloneExample}
\end{listing}

\begin{justify}
Listing \ref{listing:OutputJSONStandaloneExample} shows a reduced example of the \gls{json} output report. Only two data sources are described in the example, but \path{expressionHistory}, \path{translateCache} and \path{clipboard} are also available. Additionally, \path{rootPath} describes the root path, in case a full analysis is performed and data sources can have multiple objects in case a multiple path analysis is performed, although the same format used for both types of analysis to naturally simplify the result parsing and the overall application maintenance.
\end{justify}

\subsubsection{HTML Type}

\begin{justify}
HTML is a widely used markup language standardized for web applications to render text and images. Sometimes forensics analyzers wants a detailed custom report for more in-depth analysis. This format is useful to display textual and graphical information in a pleasant way and it is used by this project to generate a printable interactive report.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=1.0\linewidth]{imgs/GBoardReport}
 \caption{Generated report of a GBoard analysis}
 \label{fig:GBoardGeneratedHTMLReport}
\end{figure}

\begin{justify}
Figure \ref{fig:GBoardGeneratedHTMLReport} shows an example output of a generated report from a GBoard analysis.
\end{justify}

\begin{justify}
This report is easily searchable, responsive, so it fits on every screen, it is adjustable for printing purposes, by expanding all the scrollable boxes and it is portable, so it has everything bundled in a single file. Each component is designed to be opened and saved individually by expand it with a button present in the right corner and additionally, every chart can be saved in \gls{svg}, \gls{png} or \gls{csv}. This is designed specifically to help the forensic analyst to have a good user experience and be able to export information quickly.
\end{justify}

\begin{justify}
Firstly, on the left, it has a table with some brief information about the analyzed content that includes the date of the performed analysis, the full path of the GBoard root folder, if it exists and a counter with the number of items found. On the right, it has another table that describes if a certain capability was found.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=0.8\linewidth]{imgs/ReportTimeline}
 \caption{Example of a generated timeline}
 \label{fig:GBoardReportTimeline}
\end{figure}

\begin{justify}
Figure \ref{fig:GBoardReportTimeline} shows the scrollable timeline with a navigation feature to shrink and expand it by zooming in and out, moving it around with the "hand" icon, restore the default view with the "home" icon and an hamburger menu to export it. All this features can be used with common keyboard and mouse shortcuts and also the buttons present in the right corner. This is useful to seek when the victim was more activity and compare side-by-side each found capability to see what was used at some point in time.
\end{justify}

\begin{justify}
For each capability, one or more tables is shown. On the left of each table, a little question mark is present so the forensic analyst can hover and read more about what is the propose of that table. On the right, a button is present to expand the table in a new window.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=0.8\linewidth]{imgs/ReportTranslationCacheTable}
 \caption{Example of the Translation cache table}
 \label{fig:GBoardReportTranslationCacheTable}
\end{figure}

\begin{justify}
Figure \ref{fig:GBoardReportTranslationCacheTable} shows an example of the generated table for translation cache with the described features.
\end{justify}

\begin{justify}
For easier visualization of the expression history, a histogram chart is provided, so the forensic analyst can visually compare what is the most shared emoji/emoticon and therefore evaluate at first a glance if the conversation is a more friendly one or not.
\end{justify}

\begin{justify}
In the chart, it is possible to hover and see the corresponding value of the selected entry and on the top corner, a hamburger menu is present to export the chart.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=1.0\linewidth]{imgs/ExpressionHistoryChartSorted}
 \caption{Example of a generated histogram chart}
 \label{fig:GBoardReportHistogramChart}
\end{figure}

\begin{justify}
Figure \ref{fig:GBoardReportHistogramChart} shows an example of a generated histogram chart based on the Expression History information.
\end{justify}

\subsection{Internal architecture}

\begin{justify}
To achieve easier modularization and code scalability, we decided to divide the program in four modules: the gatherers, the models, the reporters and the analysis module. This way, it is possible to write features without disrupting the entire program design. Separating this logic from the rest of the code is crucial to add support for future versions of the application.
\end{justify}

\begin{justify}
The gatherers module contains all the gatherers for each implemented data source. A gatherer contains logic to fetch data from a specific data source. For example, the training cache data source has a corresponding \path{TrainingCacheGatherer} that fetches everything related to training cache related databases.
\end{justify}

\begin{justify}
The models module contains the corresponding serializable \gls{dto} structures for each data source.
\end{justify}

\begin{justify}
The reporters module has the logic to generate each implemented output report of the analyzed data.
\end{justify}

\begin{justify}
The analysis module is responsible for detecting data sources, load the appropriate data source gatherer and merging the gathered information into a serializable structure. This module is seen as a bridge for the models and the gatherers. File detection includes detecting files by their payload signature and/or check their internal structure. This module is useful to detect different file versions in newer or older versions of GBoard. This way, file detection is separated from the gatherers.
\end{justify}

\begin{listing}[ht]
\begin{textcode}
00000000: 5351 4c69 7465 2066 6f72 6d61 7420 3300  SQLite format 3.
00000010: 1000 0101 0040 2020 0000 0012 0000 0005  .....@  ........
00000020: 0000 0000 0000 0000 0000 0002 0000 0004  ................
00000030: 0000 0000 0000 0005 0000 0001 0000 0004  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
\end{textcode}
\caption{Example of the first bytes present in a SQLite database}
\label{listing:FirstBytesSQLiteDB}
\end{listing}

\begin{justify}
For example, Listing \ref{listing:FirstBytesSQLiteDB} show the first bytes of a SQLite database. In this example the file gatherer compares the the first 16 bytes with the SQLite3 file signature to check if it is an SQLite database or not and then proceed to detect the internal structure of the database. Because every SQLite database from Android has a table with metadata, it is possible to check if it is a GBoard database by checking the existance of that table and then some additional structure, if needed.
\end{justify}

\begin{listing}[ht]
\begin{sqlcode}
SELECT name FROM sqlite_master
WHERE LOWER(type) IN ('table','view')
AND LOWER(name) = 'android_metadata'
\end{sqlcode}
\caption{Query to check existence of Android metadata table}
\label{listing:AndroidMetadataTableSQL}
\end{listing}

\begin{justify}
Listing \ref{listing:AndroidMetadataTableSQL} show the query executed to detect the \path{android_metadata} table.
\end{justify}

\section{Autopsy integration}

\begin{justify}
One of the main goals of this project is to integrate the created standalone application with the Autopsy software by creating third-party modules. For this project we developed two different modules for Autopsy: an Ingest Data Source Module and a General Report Module, both communicating with the standalone application.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=1.0\linewidth]{imgs/autopsy-integration}
 \caption{Sequence diagram of the Autopsy modules}
 \label{fig:GBoardIntegrationDiagram}
\end{figure}

\begin{justify}
Figure \ref{fig:GBoardIntegrationDiagram} shows a sequence diagram of the Autopsy modules interaction with the standalone application.
\end{justify}

\subsection{Ingest Data Source Module}

\begin{justify}
The Ingest Data Source Module is intended to publish the data gathered by the standalone application to the blackboard functionality of the Autopsy. To do this, we used the generated \gls{json} output of the standalone application as a bridge format and parsed it on the Jython side.
\end{justify}

\begin{justify}
In order to publish the information on the blackboard, we needed to first create some custom artifacts to fit our extracted content, except for the clipboard content, which the Autopsy already has a built-in blackboard type. Artifacts such as Assembled timeline, emojis and emoticons expression history, history timeline, processed history, translation cache and personal dictionary were created. Those blackboard artifacts directly match the previously created models in the standalone application.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=1.0\linewidth]{imgs/AutopsyIngestModuleGBoardWide}
 \caption[Extracted content from the GBoard Autopsy Ingest Module]{Example of extracted content generated by the GBoard Autopsy Ingest Module}
 \label{fig:AutopsyIngestModuleGboard}
\end{figure}

\begin{justify}
Figure \ref{fig:AutopsyIngestModuleGboard} shows an example ouput of extracted content generated by the GBoard Autopsy Ingest Data Source Module on a full analysis.
\end{justify}

\subsection{General Report Module}

\begin{justify}
The General Report Module is intended to generate a report of any kind. In our case, this module generates an \gls{html} formatted report based on the information gathered by the standalone application. To generate this report, we simply call the standalone application using the \gls{html} reporter. Unlike Ingest Data Source Module, this module doesn't parse the output of the standalone application and rather directly maps it to a file specified by Autopsy.
\end{justify}

\section{Tests and Use Cases}

\begin{justify}
During the development of this project various tests were performed to guarantee reliability and stability across versions both for GBoard data gathering and for our standalone application behaviour. To achieve this we made manual and automated tests continuously during the development.
\end{justify}

\subsection{Manual testing environment}

\begin{justify}
All the development of this work was made primarily in Linux, using Arch Linux, although, some tests were made in Windows to garantee that our tool runs as expected with the Autopsy integration. We couldn't test manually macOS as the release for Autopsy in \path{brew} package manager was very outdated.
\end{justify}

\begin{justify}
Initially, our main testing environment were based on the docker we created\footnote{\url{https://gist.github.com/ljmf00/5db17e4cca66e762d51a86bae0712652}} with Android Emulator bundled with an Android 30 \gls{api}-based image. This image, prebuilt by Google, ships a preload version of GBoard. During the development of the project, we discovered that this image unable the installation of new versions of GBoard because the application signature of preload versions missmatch with the release branch.
\end{justify}

\begin{justify}
To proceed with this we needed to either make the \path{/product} android partition writable or have a privileged image with root access bundled with Google \gls{api}s to have the release version of GBoard. None of the above seem to be practical and we decided to move in a different direction by leaving Android Emulator and rooting our own devices.
\end{justify}

\begin{justify}
With the devices properly rooted with the Magisk software\footnote{\url{https://github.com/topjohnwu/Magisk}}, it was now possible to install different versions of GBoard. This was a good choice, as a physical device increased the number of collected data and simplified the manipulation of GBoard because the device was been extensively used. For completion, some versions of GBoard was also tested on Genymotion.
\end{justify}

\subsection{Automated Continuous Integration}

\begin{justify}
To extensively test the internal and external behaviour of our standalone application we decided to implement an automated test suite in our development pipeline. To do this, we used the recent tool from Github, called Github Actions\footnote{\url{https://docs.github.com/en/actions}}. This tool allow developers to create custom \gls{ci}/\gls{cd} workflows to automate the software development workflow and provide better software quality. \parencite{GithubActionsIntro}
\end{justify}

\begin{justify}
This allowed us to automate our \gls{ci}/\gls{cd} pipeline by compiling the code, running our test suite and deploy artifacts automatically in Windows, Linux and macOS environments, at the same time. Combining this with a \gls{pr} development strategy we continuously tested our application when adding new features or fixing bad application behaviour.
\end{justify}

\begin{justify}
Unfortunately, Autopsy doesn't provide an easy way to properly do automatic integration testing with third-party modules, so we didn't consider implementing that.
\end{justify}

\subsection{Results}

\begin{justify}
In Chapter \ref{ch:gboard} we briefly described the results of our tested GBoard versions, especially in section \ref{gboard-tested-versions} the detailed testing environment. For this project, we fetched, if available, information from clipboard management, training cache, translation cache, expression history and personal dictionary.
\end{justify}

\begin{table}[H]
\centering
\begin{threeparttable}
\begin{tabular}{|l|l|l|l|l|l|}
\hline
   & \textbf{Translation Cache} & \textbf{TC}\tnote{1} & \textbf{Clipboard} & \textbf{EH}\tnote{2} & \textbf{PD}\tnote{3} \\ \hline
9.4.11.312687073  & Yes & Yes & Yes & No & Yes \\
10.0.02.338070508 & Yes & Yes & Yes & Yes & Yes \\
10.2.07.351353117 & No & Yes & Yes & Yes & Yes \\
10.4.04.361808908 & No & Yes & Yes & Yes & Yes  \\
10.5.03.367007960 & No & No & Yes & Yes & Yes \\
\hline
\end{tabular}
\begin{tablenotes}\scriptsize
      \item[1] Training Cache
      \item[2] Expression History
      \item[3] Personal Dictionary
    \end{tablenotes}
\caption{Gboard -- tested capabilities by version}
\label{table:GboardTestedVersionsCapabilities}
\end{threeparttable}
\end{table}

\begin{justify}
Table \ref{table:GboardTestedVersionsCapabilities} shows the tested capabilities by each GBoard version previously enumerated.
\end{justify}

\begin{justify}
Some versions were more extensively tested and produced more results, but this information may vary from user to user. More active users may produce better results, although, if the user happens to have third-party application to periodically clean the applications cache or manually force cleaning it, some information is gone. If this is the case, for example, translation cache may not be captured.
\end{justify}

\begin{justify}
Contrary to what we observed, training cache is only wiped on data cleaning, rather than cache cleaning. Although, this is still considered a cache, due to the ring buffer behaviour explained previously and the fact that this database is limited to 1500 entries. In practice, the forensic analyst will only get the most recent 1500 results from that capability.
\end{justify}

\begin{justify}
The other application capabilities seem to consider application data. If the user deletes application data, because it also includes deleting cache, everything valuable, forensically speaking, is lost, unfortunately.
\end{justify}

\subsection{Real-world Results}

\begin{justify}
To present more accurate and real-world results, we decided to extensively test GBoard for three months with daily usage and more normal conversations providing a more natural and effective usage of the keyboard. In total, 6379 entries has been collected.
\end{justify}

\begin{justify}
Despite the content having the same structure, we detected some anomalies on the expression history behaviour. For a long run, expression history started do duplicate entries in the database, which leads to two emojis with different numbers in shares.
\end{justify}

\begin{figure}[H]
 \centering
 \includegraphics[width=1.0\linewidth]{imgs/RealWorldExpressionHistoryChartSorted}
 \caption{Real-world example of an expression history chart}
 \label{fig:RealWorldExpressionHistoryChart}
\end{figure}

\begin{justify}
Figure \ref{fig:RealWorldExpressionHistoryChart} shows a real-world example of an expression history chart. As shown, it is clear that the user uses a certain emoji very frequently. Moreover, it is possible to quickly understand that conversations are highly expressive and emotional.
\end{justify}

\begin{justify}
It is interesting to see that every information from the beginning of the installation is retained, except the training cache, as expected. The application or Android itself seems to not have a mechanism to internally wipe cache or old information periodically, especially the translation cache, which is specifically in the \path{cache/} folder. Although, it needs more time to effectively prove that.
\end{justify}

\begin{justify}
Another interesting fact discovered only with real-world usage, is the fact that the user seems to delete and move the cursor a lot. This is a natural behaviour in a keyboard like GBoard, as people tend to write a lot of words incorrectly, by mistake. Sometimes, incorrect order is also present, as timestamps in the training cache are not so accurate. The combination with processed history and assembled timeline is very useful for these situations.
\end{justify}