Jetpack paging library introduces a hassle-free way to implement pagination for a large set of data. Paging 3 is the latest iteration of the library and a few months ago it released the stable version. In this article, we will look into how to use this library to consume large data set over REST API. This project is build using MVVM architecture. For network call integration I’m using Retrofit with Moshi.
In MVVM architecture each layer has different responsibilities. Paging 3 library has different components which align with these architecture layers. Take a look at the below diagram.
In general, when we are dealing with remote data sources the data retrieving part is handled in the repository layer. When using the paging library there is a separate class named PagingSource which takes the responsibility of retrieving data. After creating a class that implements PagingSource class, in the repository layer all we have to do is initiate that class and wrap it in a data type that can be observe by the view model. In this example, we are using Flow but you have the freedom to use LiveData, or RxJava Flowable or Observable. In the view model simply expose this page returned by the repository while enabling caching. The paging library provides a new paging adapter called PagingDataAdapter to implement the recycler view.
In this application details of a file list is displayed in a RecyclerView. I have implemented a OkHttp interceptor to emulate the server. This interceptor is design to return random network errors to showcase the error handling capabilities in this library. I’m not going to explain that part in this article because it’s not necessary to understand paging library. You can access full repo here.
PagingSource
This class required two generic types to pass in the implementation. As the First parameter, I have passed the type of Int which represents the page number. The second parameter expects us to define the type of data load by the paging source. In our case, it’s the data class type of FileInformation. Implementation of this class is required to override two functions. The load() function is responsible for doing the network call and retrieving the data from the network. Since the load() function is a suspend function network call can be directly called here. This function needs to return LoadResult object base on network call results. If the network call is successfully returned the result should be returned as a LoadResult.Page object which contains data to showcase in the list and pagination previous and next page values. If network call fails LoadResult.Error must be returned. The getRefreshKey() function is used to get the page key and pass it into load() function when the data is refreshed or invalidated after the initial load.
private const val STARTING_PAGE_INDEX = 1
const val PAGE_ITEM_LIMIT = 10
class FileListPagingSource (private val apiService: ApiService) : PagingSource<Int, FileInformation>() {
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, FileInformation> {
try {
val position = params.key ?: STARTING_PAGE_INDEX
val postsResponse = apiService.getFiles(position, PAGE_ITEM_LIMIT)
if (postsResponse.isSuccessful && postsResponse.body() != null) {
val responseBody = postsResponse.body()
val fileInformationList = mutableListOf<FileInformation>()
if(responseBody != null) {
if(responseBody.isSuccess){
responseBody.data?.forEach {
val fileInformation = FileInformation(it.name, it.fileType, it.modifiedDate)
fileInformationList.add(fileInformation)
}
val totalRetrievedItems = (position) * PAGE_ITEM_LIMIT
return LoadResult.Page(
data = fileInformationList,
prevKey = if (position == 1) null else position - 1,
nextKey = if (totalRetrievedItems < responseBody.total) responseBody.page.plus(1) else null
)
}else{
return LoadResult.Error(
Throwable(responseBody.message, Throwable("Response Error"))
)
}
}else{
return LoadResult.Error(
Throwable("Empty Response", Throwable("Response Error"))
)
}
} else {
return LoadResult.Error(
Throwable(postsResponse.message(), Throwable("Connection Error"))
)
}
} catch (throwable: Throwable) {
val errorThrowable = Throwable(
"Something Went Wrong Please Try again later.",
Throwable("Processing Error")
)
return LoadResult.Error(errorThrowable)
}
}
override fun getRefreshKey(state: PagingState<Int, FileInformation>): Int? {
return state.anchorPosition?.let { anchorPosition ->
val anchorPage = state.closestPageToPosition(anchorPosition)
anchorPage?.prevKey?.plus(1) ?: anchorPage?.nextKey?.minus(1)
}
}
}In this example, our REST API is expecting page number and limit as the parameters. Base on the API result previous and next page key values are calculated in the load() function. To find out whether the list has reached the end of the list total retrieved Items are calculated and compared against total available items. Inside getRefreshKey() all we have to do is return the refresh page index by getting the anchor position from the PagingState and decided the refresh page base on the previous key or next key. Since the previous and next keys can be null it should be taken into consideration.
Repository
In repository simply you have to create Pager from this page source and return it as a flow.
class FileListRepository @Inject constructor(private val apiService: ApiService): BaseRepository() {
fun getFileList(): Flow<PagingData<FileInformation>> {
return Pager(
config = PagingConfig(pageSize = PAGE_ITEM_LIMIT, enablePlaceholders = false),
pagingSourceFactory = { FileListPagingSource(apiService) }
).flow
}
}Pager object required to provide configuration for the paging implementation which we can provide page size and whether we include a placeholder.
ViewModel
In the view model layer we observer the above implemented Pager flow in getFileList() repository function.
class FileListViewModel @Inject constructor(private val repository: FileListRepository)
: BaseViewModel(repository) {
fun getFileDataStream(): Flow<PagingData<FileInformation>> {
return repository.getFileList().cachedIn(viewModelScope)
}
}The cachedIn function used above makes the loaded data stream cacheable with in the provided coroutine scope.
PagingDataAdapter
Instead of regular RecyclerView adapter pagination library provides PagingDataAdapter for easy pagination implementation. Because of this, we do not need to implement on scroll listener which we used in traditional pagination implementation.
class FileListAdapter(private val onClickFile:(FileInformation?) -> Unit)
: PagingDataAdapter<FileInformation,FileListViewHolder>(itemComparator) {
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): FileListViewHolder {
return FileListViewHolder.create(parent,onClickFile)
}
override fun onBindViewHolder(holder: FileListViewHolder, position: Int) {
val item = getItem(position)
if(item != null){
holder.bindRepoData(item)
}
}
companion object {
private val itemComparator = object : DiffUtil.ItemCallback<FileInformation>() {
override fun areItemsTheSame(oldItem: FileInformation, newItem: FileInformation): Boolean {
return oldItem.name == newItem.name
}
override fun areContentsTheSame(oldItem: FileInformation, newItem: FileInformation): Boolean {
return oldItem == newItem
}
}
}
}PagingDataAdapter implementation is expected to provide two generic type parameters. The first parameter defines the type of data object which use to showcase in the list view. The second parameter type defines the RecyclerView.ViewHolder type which use to create RecyclerView list item. As you can see this class implementation is not that different from RecyclerView.Adapte implementation. We have familiar onCreateViewHolder and onBindViewHolder functions to provide the view holder and bind the data. I have created a view holder in a separate class which you can find in the repo. In addition to that DiffUtil implementation is required by PagingDataAdapter to have good performance in recycler view.
Fragment
At this point, we have created all the paging library-related classes to create the RecyclerView with pagination. Now we will bring everything together in the view layer.
private fun initialization() {
fragmentSubComponent = injector.fragmentComponent().create(requireView())
fragmentSubComponent.inject(this)
val fileListAdapter = FileListAdapter { onClickProfile(it) }
viewBinding.filesRecyclerView.layoutManager = LinearLayoutManager(requireContext())
viewBinding.filesRecyclerView.adapter = fileListAdapter
lifecycleScope.launch {
viewModel.getFileDataStream().collectLatest{
fileListAdapter.submitData(it)
}
}
super.initialization(null,null)
}This is the initialization() function implementation which is called inside onViewCreated() function inside the fragment. Setting up the adapter to the recycler view is the same way as setting up a general RecyclerView. But the only difference here is how the data is supplied for the Adapter. PagingDataAdapter has a function called submitData() to provide the necessary data for the adapter. Those data can be observed by the from getFileDataStream() view model function by calling collectLatest on it. Paging library will automatically request the data on demand to fill the RecyclerView when more data is available in PagingSource.
Displaying Loading Footer and Error Messages
For a better user experience when using paging implementation we need to indicate to the user that the items are been loading. In case REST API fails to deliver the data we need to indicate a retry option. With the Paging library we can easily implement these functionalities.
The paging library provides a LoadStateAdapter class to implement which we can use to show the loading footer. Same as the PagingDataAdapter we can provide a RecyclerView.ViewHolder for this to use our own design.
class FileLoadStateAdapter (private val retry: () -> Unit)
: LoadStateAdapter<FileLoadStateViewHolder>(){
override fun onBindViewHolder(holder: FileLoadStateViewHolder, loadState: LoadState) {
holder.bind(loadState)
}
override fun onCreateViewHolder(parent: ViewGroup, loadState: LoadState): FileLoadStateViewHolder {
return FileLoadStateViewHolder.create(parent, retry)
}
}LoadStateAdapter required a simple implementation which we are required to implement only onBindViewHolder() and onCreateViewHolder() which are responsible for drawing custom footer item view into recycler view and binding data into it. The retry function is provided from the fragment to give the user ability to retry the network call if it fails. After completing this implementation in fragment you need to supply this FileLoadStateAdapter as a Footer. The fragment initialization() function should be modified like below.
private fun initialization() {
fragmentSubComponent = injector.fragmentComponent().create(requireView())
fragmentSubComponent.inject(this)
val fileListAdapter = FileListAdapter { onClickProfile(it) }
viewBinding.filesRecyclerView.layoutManager = LinearLayoutManager(requireContext())
viewBinding.filesRecyclerView.adapter = fileListAdapter.withLoadStateFooter(
FileLoadStateAdapter { fileListAdapter.retry() }
)
lifecycleScope.launch {
viewModel.getFileDataStream().collectLatest{
fileListAdapter.submitData(it)
}
}
super.initialization(null,null)
}Displaying Loading State in Parent View
On initial data load loading footer can not be displayed since we don’t have any data in the RecyclerView . In this scenario, we need to show the loading animation or the message in the parent activity view. to do that we need to add the view to the parent view like below.
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".ui.fragments.FileListFragment">
<androidx.recyclerview.widget.RecyclerView
android:id="@+id/filesRecyclerView"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_centerInParent="true"/>
<ProgressBar
android:id="@+id/progressBar"
style="?android:attr/progressBarStyle"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_centerInParent="true"/>
<LinearLayout
android:id="@+id/retryLayoutContainer"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:orientation="vertical"
android:layout_centerInParent="true">
<TextView
android:id="@+id/errorMessage"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center"
android:layout_marginStart="16dp"
android:layout_marginEnd="16dp"
android:textAlignment="center"
android:textColor="?android:textColorPrimary"
android:textSize="14sp"
tools:text="Network error"
android:gravity="center_horizontal"/>
<Button
android:id="@+id/retryButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center"
android:layout_marginTop="8dp"
android:text="@string/retry_button_label" />
</LinearLayout>
</RelativeLayout>After that in our fragment we need to implement LoadStateListener for our adapter.
fileListAdapter.addLoadStateListener { loadState ->
val isListEmpty = loadState.refresh is LoadState.NotLoading && fileListAdapter.itemCount == 0
if(isListEmpty){
viewBinding.retryLayoutContainer.visibility = View.VISIBLE
viewBinding.errorMessage.visibility = View.VISIBLE
viewBinding.retryButton.visibility = View.GONE
viewBinding.errorMessage.text = getString(R.string.no_results)
}else{
viewBinding.retryLayoutContainer.visibility = View.GONE
viewBinding.errorMessage.visibility = View.VISIBLE
viewBinding.retryButton.visibility = View.VISIBLE
}
viewBinding.filesRecyclerView.isVisible = loadState.source.refresh is LoadState.NotLoading
viewBinding.progressBar.isVisible = loadState.source.refresh is LoadState.Loading
viewBinding.retryLayoutContainer.isVisible = loadState.source.refresh is LoadState.Error
val errorState = loadState.source.append as? LoadState.Error
?: loadState.source.prepend as? LoadState.Error
?: loadState.append as? LoadState.Error
?: loadState.prepend as? LoadState.Error
errorState?.let {
viewBinding.errorMessage.text = "\uD83D\uDE28 Wooops ${it.error}"
}
}In this implementation simply check whether the list is empty and base on that showing the appropriate UI elements. After that base on the loadState.source.refresh value loading state is decided. if the loadState.source.refresh value is not loading RecyclerView is made visible and the retry view container is made invisible. If the value of the loadState.source.refresh is loading progress bar is made visible while other elements are hidden. Finally if loadState.source.refresh value is an error retry layout container is made visible while other elements are invisible. Finally by checking whether CombinedLoadStates.prepend or CombinedLoadStates.append is an instance of LoadState.Error and we can display the appropriate error message provided from FileListPagingSource
This concludes the implementation of the Paging 3 library which guided you to consume large data sources over the network. The paging library also has a feature to work with SQLite database to show data while having a database cache. This functionality is not production-ready yet. I’m hoping to do an article about that in the future.
