Rollups

One of the ways that jAgg supports super aggregation is by supporting rollups. Normally, jAgg calculates aggregate values based on all "group by" properties specified in the Builder object. However, sometimes different levels of aggregation are desired. For example, subtotals for each group of aggregate values can be desirable.

Once can simply call jAgg again, with less properties, to obtain the desired subtotals, but that would mean a separate pass through the data for each level of super aggregation. With rollups, and super aggregation in general, jAgg reuses the Aggregators that were used to calculate the original, normal aggregate values, in order to calculate the new super aggregate values.

Understanding Rollups

When rollups are specified, jAgg needs to know which properties by which to create subtotals. jAgg expects 0-based indices to specify which properties. These indices refer to the original list of property names that was supplied to the Builder object. jAgg will create subtotals first at the last specified property, then it will roll up those subtotals to the property specified before that last property, and so on, until all rollup properties have been subtotaled. Unlike cube, a rollup of n properties produces only n extra grouping set combinations.

In this example, four property names were originally specified to jAgg. Without rollups, normal aggregation proceeds and here are the results:

List<String> properties = Arrays.asList("property1", "property2", "property3", "property4");
List<Aggregator> aggs = Arrays.asList(Aggregator.getAggregator("Sum(value)"));
Aggregation agg = new Aggregation.Builder()
   .setProperties(properties)
   .setAggregators(aggs)
   .build();
List<AggregateValue<TestRec>> aggValues = agg.groupBy(testRecords);
            
property1 property2 property3 property4 Sum(value)
A 1 red true 2
A 1 red false 3
A 1 green true 6
A 1 green false 10
A 2 red true 102
A 2 red false 103
A 2 green true 106
A 2 green false 110
B 1 red true 1002
B 1 red false 1003
B 1 green true 1006
B 1 green false 1010
B 2 red true 1102
B 2 red false 1103
B 2 green true 1106
B 2 green false 1110

Here are the new results when the following rollups are specified: {1, 2, 3}, corresponding to "property2", "property3", and "property4", respectively.

List<String> properties = Arrays.asList("property1", "property2", "property3", "property4");
List<Aggregator> aggs = Arrays.asList(Aggregator.getAggregator("Sum(value)"));
List<Integer> rollupProps = Arrays.asList(1, 2, 3);
Aggregation agg = new Aggregation.Builder()
   .setProperties(properties)
   .setAggregators(aggs)
   .setRollup(rollupProps)
   .build();
List<AggregateValue<TestRec>> aggValues = agg.groupBy(testRecords);
            
property1 property2 property3 property4 Sum(value)
A 1 red true 2
A 1 red false 3
A 1 green true 6
A 1 green false 10
A 2 red true 102
A 2 red false 103
A 2 green true 106
A 2 green false 110
B 1 red true 1002
B 1 red false 1003
B 1 green true 1006
B 1 green false 1010
B 2 red true 1102
B 2 red false 1103
B 2 green true 1106
B 2 green false 1110
A 1 red   5
A 1 green   16
A 2 red   205
A 2 green   216
B 1 red   2005
B 1 green   2016
B 2 red   2205
B 2 green   2216
A 1     21
A 2     421
B 1     4021
B 2     4421
A       442
B       8442

Subtotals are calculated at increasingly general levels. Note here that the order of the rollup properties specified is important, because the properties referenced last will get rolled up before the properties referenced first. Also, whenever a property is rolled up, its value is null, meaning that the particular aggregate value represents "all values" for this property.

Above, every property was subtotaled except for "property1", because "0" was not specified. Here is another example, including "property1". Because all properties are now specified, there is one AggregateValue where getPropertyValue always returns null. This is the grand total.

List<String> properties = Arrays.asList("property1", "property2", "property3", "property4");
List<Aggregator> aggs = Arrays.asList(Aggregator.getAggregator("Sum(value)"));
List<Integer> rollupProps = Arrays.asList(0, 1, 2, 3);
Aggregation agg = new Aggregation.Builder()
   .setProperties(properties)
   .setAggregators(aggs)
   .setRollup(rollupProps)
   .build();
List<AggregateValue<TestRec>> aggValues = agg.groupBy(testRecords);
            
property1 property2 property3 property4 Sum(value)
A 1 red true 2
A 1 red false 3
A 1 green true 6
A 1 green false 10
A 2 red true 102
A 2 red false 103
A 2 green true 106
A 2 green false 110
B 1 red true 1002
B 1 red false 1003
B 1 green true 1006
B 1 green false 1010
B 2 red true 1102
B 2 red false 1103
B 2 green true 1106
B 2 green false 1110
A 1 red   5
A 1 green   16
A 2 red   205
A 2 green   216
B 1 red   2005
B 1 green   2016
B 2 red   2205
B 2 green   2216
A 1     21
A 2     421
B 1     4021
B 2     4421
A       442
B       8442
        8884

Identifying Grouping Sets

If a certain property represents "all values", then the result from getPropertyValue for that property will be null. But what if null is the actual value being aggregated? jAgg tells these cases apart with the help of the methods isGrouping() and getGroupingId.

  • isGrouping(int field) - Determines whether the property referenced by the given 0-based index represents "all values". If true, then getPropertyValue(field) returns null and this is a super aggregate value. If false, then getPropertyValue(field) can return any value, including null, and this aggregate value does not represent "all values" for this property.
  • isGrouping(String propertyName) - Determines whether the given property represents "all values". If true, then getPropertyValue(field) returns null and this is a super aggregate value. If false, then getPropertyValue(field) can return any value, including null, and this aggregate value does not represent "all values" for this property.
  • getGroupingId(List<?> fields) - Creates a distinct integer grouping set ID based on the referenced fields, which may be 0-based integer references or property name strings, or both. Every aggregate value that has the same properties representing "all values" has the same integer ID.

Here is the same example as above, but with the above method call results included.

property1 property2 property3 property4 Sum(value) isGrouping(0) isGrouping(1) isGrouping(2) isGrouping(3) getGroupingId({0, 1}) getGroupingId({0, 1, 2, 3})
A 1 red true 2 false false false false 0 0
A 1 red false 3 false false false false 0 0
A 1 green true 6 false false false false 0 0
A 1 green false 10 false false false false 0 0
A 2 red true 102 false false false false 0 0
A 2 red false 103 false false false false 0 0
A 2 green true 106 false false false false 0 0
A 2 green false 110 false false false false 0 0
B 1 red true 1002 false false false false 0 0
B 1 red false 1003 false false false false 0 0
B 1 green true 1006 false false false false 0 0
B 1 green false 1010 false false false false 0 0
B 2 red true 1102 false false false false 0 0
B 2 red false 1103 false false false false 0 0
B 2 green true 1106 false false false false 0 0
B 2 green false 1110 false false false false 0 0
A 1 red   5 false false false true 0 1
A 1 green   16 false false false true 0 1
A 2 red   205 false false false true 0 1
A 2 green   216 false false false true 0 1
B 1 red   2005 false false false true 0 1
B 1 green   2016 false false false true 0 1
B 2 red   2205 false false false true 0 1
B 2 green   2216 false false false true 0 1
A 1     21 false false true true 0 3
A 2     421 false false true true 0 3
B 1     4021 false false true true 0 3
B 2     4421 false false true true 0 3
A       442 false true true true 1 7
B       8442 false true true true 1 7
        8884 true true true true 3 15

Multiple Rollups

Multiple groups of rollups are possible. Each rollup combination is itself rolled up with every other rollup combination.

List<String> properties = Arrays.asList("property1", "property2", "property3", "property4");
List<Aggregator> aggs = Arrays.asList(Aggregator.getAggregator("Sum(value)"));
List<List<Integer>> rollupsProps = Arrays.asList(
    Arrays.asList(0, 1),
    Arrays.asList(2));
Aggregation agg = new Aggregation.Builder()
   .setProperties(properties)
   .setAggregators(aggs)
   .setRollups(rollupsProps)
   .build();
List<AggregateValue<TestRec>> aggValues = agg.groupBy(testRecords);
            
property1 property2 property3 property4 Sum(value)
A 1 red true 2
A 1 red false 3
A 1 green true 6
A 1 green false 10
A 2 red true 102
A 2 red false 103
A 2 green true 106
A 2 green false 110
B 1 red true 1002
B 1 red false 1003
B 1 green true 1006
B 1 green false 1010
B 2 red true 1102
B 2 red false 1103
B 2 green true 1106
B 2 green false 1110
A   red true 104
A   red false 106
A   green true 112
A   green false 120
B   red true 2104
B   red false 2106
B   green true 2112
B   green false 2120
    red true 2208
    red false 2212
    green true 2224
    green false 2240
A 1   true 8
A 1   false 13
A 2   true 208
A 2   false 213
B 1   true 2008
B 1   false 2013
B 2   true 2208
B 2   false 2213
A     true 216
A     false 226
B     true 4216
B     false 4226
      true 4432
      false 4452