DAS Analytics classes¶

Analytics cli module¶

Analytics config module¶

Analytics controller module¶

Analytics schedule module¶

Analytics task module¶

Analytics utils module¶

Analytics web module¶

Analytics task module¶

Analytics HotspotBase module¶

Base class for scheduling analyzers

class DAS.analytics.tasks.hotspot_base.HotspotBase(**kwargs)¶

This is a base-class for periodically-running analyzers that want to examine the moving average of some key->counter map, and pick the top few for further attention.

DASQueries are extracted from analytics DB. The selected items are passed to generate_task callback implemented in subclasses. It look-up DAS query expiration timestamp and if necessary calls DAS to get it (along with results of the query).

generate_task(item, count, epoch_start, epoch_end)¶

For the given selected key, generate an appropriate task dictionary as understood by taskscheduler.

Should be a generator or return an iterable

get_all_items(summaries)¶: Merge the summary dictionaries.

get_summaries(epoch_start, epoch_end)¶: Fetch all the available pre-computed summaries and determine if any need to be constructed at this time.

make_one_summary(start, finish)¶: Actually make a summary of item->count pairs for the specified time range. Subclasses need to implement this for the analysis in question.

make_summary(start, finish)¶: Split the summarise requests into interval-sized chunks and decide if they’re necessary at all.

mutate_items(items)¶

This is a last part of selection chain.

Optionally, mutate the selected items. A subclass wishing to merge together keys should do so here.

preselect_items(items)¶

This is a part of selection chain.

Optionally, preselect the items for consideration. A subclass wishing to exclude certain key types could do so here (but could also do so in make_one_summary).

This is a good place to implement clustering algorithm for selected items. For example, if several queries are selected, we may analyze who has more weight and only pass those for task generation step.

report()¶: Generate some extra keys to go in the job report, if desired.

select_items(items)¶: Take a mapping of item->count pairs and determine which are “hot” based on the selected mode.

Analytics KeyHotspot module¶

KeyHotspot analyzer

class DAS.analytics.tasks.key_hotspot.KeyHotspot(**kwargs)¶

This analyzer runs infrequently (~daily) to determine based on a moving average of the whole period which keys (not their values) are most requested, and uses this to start several child ValueHotspot instances to monitor those keys, using the HotspotBase metrics.

generate_task(item, count, epoch_start, epoch_end)¶: Generate task

DAS, version development