Filter data

Filter data before its sent over the internet (to reduce the data size sent)

You can filter the data using a query option while fetching the data. This is more efficient than sending the whole database over the internet when you only want a small section of it.

This example below shows you how to do this when requesting data from a Fliplet Data Source. If you’re retrieving data from a third party database then you’ll need to follow their documentation.

The data you get back from a Fliplet Data Dource will contain the following (as an array of Objects):

• id
• data
• order
• createdAt
• updatedAt
• deletedAt
• dataSourceId

The code below connects to a data source with ID of ‘87956’ (you can find the ID in the App Data section of Fliplet Studio).

Fliplet.DataSources.connect(87956)
  .then(function (connection) {
    return connection.find({
      where: {
        name: {
          $in: ['Nick', 'Tony']
        }
      }
    });
  }).then(function (records) {
    // Will return data that match the filter
  });

The above will return the entries that have the name Nick or Tony in their ‘name’ field. For more information on the different filtering options follow this.

Filter data that you already have on a device

If you already have the data on a device or if you want to do more complex filtering, Fliplet recommends that you use the lodash.js library.

It then uses the lodash (represented by ‘_’) to filter the records for entries that contain the name ‘Nick’ or the name ‘Tony’.
Lodash accepts a range of filter options, for further details on filtering options please see here.

The example below assumes we have some data (unfilteredData) that is in the same structure as a Fliplet Data Source. We then create a new data set called filteredData if the data has a field of ‘name’ and that name is equal to ‘Nick’ or ‘Tony’.

let filteredData = _.filter(unfilteredData, function (record) {
  return record.data['Name'] === 'Nick' || record.data['Name'] === 'Tony'
});

If we wanted to combine this with a data request from a Fliplet data source (ie, get the data, then filter it) we could do the following:

let filteredData = [];

Fliplet.DataSources.connect(87956)
  .then(function (connection) {
    return connection.find();
  }).then(function (records) {
    // Use lodash '_' to filter the data
    filteredData = _.filter(records, function (record) {
      return record.data['Name'] === 'Nick' || record.data['Name'] === 'Tony'
    });
  });

You should be aware that numbers may be stored as text in a Fliplet Data Source (for example if a user typed in numbers via a text input form field instead of a number input field). This can lead to some confusion when filtering and sorting. When putting letters in alphabetical order “a” comes before “aa” which comes before “b”. But for numbers “1” comes before “11” but “11” does not come before “2”. To ensure that numbers are filtered (and sorted) correctly we need to convert them from strings (aka text) into a number.

If your numbers don’t have any decimal places you can use parseInt() (find more about it here). If they do you can use parseFloat (find more about it here). In the example below, we’re using parseInt() on the age data (the ’10’ in the code denotes that we’re converting from text to a decimal number). We then filter the data if the age is greater than 35 and save the result to filteredData.

var filteredData = [];

Fliplet.DataSources.connect(87956)
  .then(function (connection) {
    return connection.find();
  }).then(function (records) {
    filteredData = _.filter(records, function(record) {
      return parseInt(record.data['Age'], 10) > 35;
    });
  });

You can also filter the data by more than one condition. The && represents an ‘AND’ and the || represents an ‘OR’. Brackets can be used to separate different conditions. So the filter below will return data if the record has a name of either Nick or Tony and that data also has an age equal to 36.

var filteredData = [];

Fliplet.DataSources.connect(1)
  .then(function (connection) {
    return connection.find();
  }).then(function (records) {
    filteredData = _.filter(records, function(record) {
      return (record.data['Name'] === 'Nick' || record.data['Name'] === 'Tony')
        && parseInt(record.data['Age'], 10) === 36;
    });
  });

Related topics

Sorting data.