Best Practices for Data Source Size

Data SourcesClosed Data sources, also known as "Lookups", are external sources of data that you upload or connect to TrueContext. You can reference this data in a form to populate answers or answer options. Data sources save typing, reduce errors, and make it easy to provide mobile users with only the relevant, most current data. make external data available to the TrueContext Mobile App. We store Data Sources as tables (rows and columns) in the TrueContext system. This topic describes best practices for organizing your data and explains why we limit the size of Data Sources by default.

Tip:If you’re an Enterprise customer, we recommend that you follow the recommendations for Data Source size before you request an increase to the maximum number of cells and rows. The process that you follow to request an increase ensures that we have an opportunity to help you optimize mobile device performance.

Contents

Info:The topic Data Sources Overview describes the use of Data Sources in more detail.

Recommendations for Data Source size

We recommend that you keep your Data Sources as small as possible for the following reasons.

Enhanced user experience
Tip:A frustrated user might complete an outdated version of a form using data that’s not current. Use smaller Data Sources to give your users quick access to your forms.
Data integrity
  • Your organization must regularly maintain and update the data that’s used in a TrueContext Data Source. Organize your data logically so that it’s easier for you to maintain. For example, it’s easier to maintain 1,000 rows of data than it is to maintain 50,000 rows.

  • Smaller Data Sources make it easier to avoid duplication and to maintain data integrity.

  • For high volumes of data, follow best practices for database normalization. This reduces the amount of data to transfer.

  • Make sure your data model is designed to maximize efficiency.

Tip:You can also partition a Data Source (Enterprise tier only) to optimize mobile device performance, but this doesn’t reduce the actual Data Source size.

Why we limit the size of your Data Sources

Data Source size limits are a way to help you optimize mobile device performance. The more data that you load on a device, the longer it takes for the device to reconcile.

The size limits apply when you upload, fetch, or update a Data Source. The system enforces a column and character limit for all tiers. Other size limits vary by tier, as shown in the following table.

Tier Maximum number of cells Maximum number of rows Maximum number of columns Maximum number of characters in a cell Maximum file size
Essentials 100,000 10,000 500 32,767 500 MB
Advanced 500,000 50,000
Enterprise 2,000,000 (default) 200,000 (default)

How Data Source size limits work

File size

The system limits the size of manually uploaded and dynamically fetched Data Source files to 500 MB. If your file exceeds the limit, the system blocks you from uploading or using the file.

Rows and cells

The TrueContext system limits the size of each Data Source based on the number of rows and cells (rows multiplied by columns). The system

  • calculates the number of rows and cells

  • compares these numbers to the limits for your tier, and

  • determines whether the Data Source exceeds the size limits.

If there are too many rows or cells, the system uploads the data but cuts off (truncates) any rows and cells that exceed the limit. In this case, the system displays an on-screen message and emails the problem contact email address.

Columns and characters

If a Data Source has more than 500 columns or 32,767 characters in one cell, regardless of tier, the system displays an error message and does not upload any data.

How to resolve a Data Source size limit issue

Remember that large Data Sources can cause a device to take longer to reconcile. This might result in a poor mobile user experience.

To resolve the issue of truncated data, try one of the following options.

  • Reduce the Data Source size by removing columns or rows.

  • Use multiple Data Sources instead of one. For example, you can separate customer data from equipment data.

  • Upgrade to a higher tier (contact your Account Manager or your Customer Success Manager for details).

Size limit increases for Enterprise customers

Tip:We recommend that you follow the recommendations for Data Source size before you request a size limit increase.

If you’re an Enterprise customer, you can contact Support to discuss a size limit increase. Please provide Support with the following information:

  • Are all columns in the data source required? Could you remove some to reduce the size?

  • Will partitions be configured? If yes, how many rows and cells will the largest partition contain?

  • What is the business reason for the cell increase?

  • How often will you need to update the Data Source?

We’ll pass this information along to our development team to evaluate the request and implement the change.