XpoLog help - Reports
 

XpoLog comes with a built in feature that allows you to create generic reports to better analyze and understand your data. In XpoLog, "report" stands for a report template - the definition of a report; generating a report results in a "report runtime". One report can have many report runtimes, each being a result of a different report generation.
Currently XpoLog supports "Aggregation" reports which analyze your data in terms of total number of occurrences and distribution over time of specific data within your logs.
There are five types of reports : case, column (or dimension), content, computation and similarity. A case is the definition of a subset of your data, similar to the notion of a filter. For instance, if you have a log with a 'priority' field and you want to create a report that shows the total number of occurrences and the distribution over time of the different priority values, such as 'info', 'warn', 'error' and 'fatal', you can define different cases, each corresponding to a different priority filter. A column (dimension) correspond to a log column. A report runtime of a report with dimension definitions holds information about that dimension's (column's) values: which values appear in that column, how often and in what distribution. For a dimension definition there's no need to define filters, as is the case in the definition of report cases. A content report automatically identifies predefined expressions that can indicate a problem and creates a report runtime that shows the total number for each of these expressions. A computation report enables you to execute functions (predefined or newly created ones) on log's fields. By using these functions one can retrieve information about the column's value like the average value of the column, the aggregated sum of the column and so on. A similarity report identifies log's similar events, so that its report runtime displays a list of log events patterns with the number of real log event comprising it. If XpoLog identifies a constant number in a column (like a IP or timestamp) it will replace it with a '<XPLG_N>' tag inside the event pattern. A constant string will be replaced with a '<XPLG>' tag.

 

Defining reports  

  1. Select the 'Reports' menu to enter the 'Reports Definition' page. This page shows all the reports in the system, grouped by the applications they belong to. Reports that are not associated with any application are all grouped under 'Other'. Click the arrow next to an application to display its reports.
  2. Select 'Add new Report' to create a new report or 'edit' of an existing report to edit it.
  3. Enter the name of the report and optionally add a description to it.
  4. In the 'Data Distribution' section, select the data unit for which aggregation data will be computed. You can override this setting when you generate a new report runtime. Chose one of the following options:
    • None: No computation of data distribution will take place. an empty cases distribution graph will be displayed
    • Automatic: XpoLog sets automatically the data units either to 'hours' or to 'days', according to the time span of the different data sources
    • Over days: data will be aggregated for each day
    • Over hours: data will be aggregated for each hour
    • Over minutes: data will be aggregated for each minute
  5. In the 'Time Filter' section you can define default time constraints for the report generation. You can override this setting when you generate a new report runtime. You can define time constraints, or filters, in two ways - similar to the way a date filter is defined:

    1. selecting the 'Dates limit' option and entering the minimum and\or maximum date
    2. selecting the 'show records' option. in the following combo box select one of the following options:

      • from the last: defines a time interval that starts with the specified time constraint and stretches until the current time
      • from the previous: defines a time interval that contains only the specified days, weeks or months
      • from: defines a time interval the starts in the data specified in the number text field and time unit (in the case of 'days' a starting hour is required too) and stretches for the time span defined in the 'for' section of the filter
  6. In the 'Status' section select the logic by which the report's status will be determined. Select 'at least one case failed' in order to set the report status to 'failed' when at least one case/dimension has failed, or 'all cases failed' in order to set the report status to 'failed' when all of the report's cases/dimensions have failed.
  7. Click 'next' to get to the report cases and columns definition screen. Here is where you define the different report cases and dimensions (columns), which are the core of each report.

  8. Click 'Add new aggregation rule' to create a new case or 'edit' of an existing case to edit it.
    If TxExpo is enabled, then the 'Cases List' page will be displayed in which you can select the type of aggregation case you want to define. A 'Log Case Aggregation' is a case based on a log and usually at least one of its filters, and a 'Transaction Case Aggregation' is a case based on a result set in TxExpo. If TxExpo is not enabled, you will be redirected to the log case aggregation page.

    Log Case Selection:

    • In the 'General' section, enter the name of the case and add optionally a description to it. If you leave the default description 'LOG_NAME [FILTERS_NAME]', XpoLog will replace in the created report 'LOG_NAME' with the name of the selected log and 'FILTERS_NAME' with the name of the selected filter(s).
    • In the 'Log List' section, select the log to be used for that case. You can either select a log from the logs tree or select a previously selected log from the 'Selected from used logs' combo box. Click 'next' to enter the 'Log Case - Fields' page.
    • In the 'Status' section, click on the check box to turn on the computation of the case's status. Select 'more than' or 'less than' and enter the number of records in the result to complete the definition of the case's status. If the checkbox is left unchecked, no status for that case will be computed.

    • In the 'Data Filters' section, select the filters that define your case - a subset of the data of the selected log. If you select multiple filters, the subset defined thus is the union of all the subsets defined by the single filters (an 'OR' operation).
    • Click 'next' to complete the case's definition and return to the 'Reports Cases' page.

    Transaction Case Selection:

    • select this option to enter the 'Transaction Case Selection' page.
    • In the 'General' section, enter the name of the case and add optionally a description to it.
      If you leave the default description 'TRANSACTION_NAME [FILTERS_NAME]', XpoLog will replace in the report result TRANSACTION_NAME with the name of the selected project and FILTERS_NAME with the name of the selected filter(s).
    • In the 'Transactions List' section, select the transaction to be used for that case. XpoLog will use the only runtime transactions of the select transaction. If no time filter was specified in the report's definition (see above), then the last result set will be used, otherwise XpoLog will scan all available result sets that match the defined time constraint. Click 'next' to enter the 'Transaction Case - Fields' page.
    • In the 'Status' section, click on the check box to enable the determination of the case's status. Select 'more than' or 'less than' to finish the definition of the case's status. If the checkbox is left unchecked, no status for that case will be determined.
    • In the 'Status Filters' section, select the statuses that participate in the definition of your runtime transaction's subset. The case will include only these transactions that have at least one of the selected status filters.
    • In the 'Data Filters' section, select the data filters that further participate in the definition of your runtime transaction's subset. The case will include only these transactions that have at least one of the selected data filters.
    • Click 'next' to complete the case's definition and return to the 'Reports Cases' page.
       
  9. Click 'Add new column aggregation' to create a new column aggregation or 'edit' of an existing column aggregation to edit it.

    • In the 'General' section, enter the name of the case and add optionally a description to it. If you leave the default description 'LOG_NAME [COLUMNS_NAME]', XpoLog will replace in the created report 'LOG_NAME' with the name of the selected log and 'COLUMNS_NAME' with the name of the selected column.

    • In the 'Log List' section, select the log to be used for that case. You can either select a log from the logs tree or select a previously selected log from the 'Selected from used logs' combo box. Click 'next' to enter the 'Log Column Aggregation - Column' page.

    • In the 'Column aggregation' section select the log's column. When the a report runtime will be generated, the different values this column can assume will be computed and aggregated over the selected time unit.
    • In the 'Data Filters' section, select the filters that should be used is fetching the log records. Only log records that comply with these filters will take part in the report's computation.

    • In the 'Table display settings' you can set the number of results to be displayed in the that column section of the report runtime result. XpoLog collects only the top 5000 values with the occurrences of different

    • Click 'next' to complete the case's definition and return to the 'Reports Cases' page.
    • In the 'Status' section, click on the check box to turn on the computation of the case's status. Select 'more than' or 'less than' and enter the number of records in the result to complete the definition of the case's status. If the checkbox is left unchecked, no status for that case will be computed.


     
  10. Click 'add new content aggregation' to create new content aggregation or 'edit' of an existing one to edit it.

    • In the 'General' section, enter the name of the content and add optionally a description to it. If you leave the default description 'LOG_NAME [content]', XpoLog will replace in the created case 'LOG_NAME' with the name of the selected log.

    • In the 'Log List' section, select the log to be used for that case. You can either select a log from the logs tree or select a previously selected log from the 'Selected from used logs' combo box. Click 'next' to enter the 'Log Content Aggregation' page.

    • In the 'Data Filters' section, select the filters that should be used is fetching the log records. Only log records that comply with these filters will take part in the report's computation.

    • In the 'Table display settings' you can set the number of results to be displayed in the that column section of the report runtime result. XpoLog collects only the top 5000 values with the occurrences of different

    • Click 'next' to complete the content's definition and return to the 'Reports Cases' page.In the 'Status' section, click on the check box to turn on the computation of the report's status. Select 'more than' or 'less than' and enter the number of records in the result to complete the definition of the report's status. If the checkbox is left unchecked, no status for that report will be computed.

     

  11. Click 'add new computation aggregation' to create new computation aggregation or 'edit' of an existing one to edit it.
    If TxExpo is enabled, then the 'Computation List' page will be displayed in which you can select the type of aggregation computation you want to define. A 'Log Computation Aggregation' is a case based on a log and usually at least one of its filters, or 'Transaction Computation Aggregation' is a case based on a result set in TxExpo. If TxExpo is not enabled, you will be redirected to the log computation aggregation page.

    Log Computation Selection:
      • In the 'General' section, enter the name of the computation and add optionally a description to it. If you leave the default description 'LOG_NAME [computation]', XpoLog will replace in the created case 'LOG_NAME' with the name of the selected log.
      • In the 'Log List' section, select the log to be used for that case. You can either select a log from the logs tree or select a previously selected log from the 'Selected from used logs' combo box. Click 'next' 
      • In the 'Function' tab you have select on which column you want to use the computation and the function to make this computation. You can also add new function by clicking the 'Add new function' link. See ' Add New Computation Function' help.
      • In the 'Status' section, click on the check box to turn on the computation of the case's status. Select 'more than' or 'less than' and enter the number of records in the result to complete the definition of the case's status. If the checkbox is left unchecked, no status for that case will be computed.

      • In the 'Data Filters' section, select the filters that define your case - a subset of the data of the selected log. If you select multiple filters, the subset defined thus is the union of all the subsets defined by the single filters (an 'OR' operation).
      • Click 'next' to complete the case's definition and return to the 'Reports Cases' page.

    Transaction Case Selection:

      • select this option to enter the 'Transaction Case Selection' page.
      • In the 'General' section, enter the name of the case and add optionally a description to it.
        If you leave the default description 'TRANSACTION_NAME [FILTERS_NAME]', XpoLog will replace in the report result TRANSACTION_NAME with the name of the selected project and FILTERS_NAME with the name of the selected filter(s).
      • In the 'Transactions List' section, select the transaction to be used for that case. XpoLog will use the only runtime transactions of the select transaction. If no time filter was specified in the report's definition (see above), then the last result set will be used, otherwise XpoLog will scan all available result sets that match the defined time constraint. Click 'next' to enter the 'Transaction Case - Fields' page.
      • In the 'Status' section, click on the check box to enable the determination of the case's status. Select 'more than' or 'less than' to finish the definition of the case's status. If the checkbox is left unchecked, no status for that case will be determined.
      • In the 'Status Filters' section, select the statuses that participate in the definition of your runtime transaction's subset. The case will include only these transactions that have at least one of the selected status filters.
      • In the 'Data Filters' section, select the data filters that further participate in the definition of your runtime transaction's subset. The case will include only these transactions that have at least one of the selected data filters.
      • Click 'next' to complete the case's definition and return to the 'Reports Cases' page.
         

  12. Click 'Add new similarity aggregation' to create a new similarity aggregation or 'edit' of an existing similarity aggregation to edit it.

    • In the 'General' section, enter the name of the case and add optionally a description to it. If you leave the default description 'LOG_NAME [similarity]', XpoLog will replace in the created report 'LOG_NAME' with the name of the selected log .

    • In the 'Log List' section, select the log to be used for that case. You can either select a log from the logs tree or select a previously selected log from the 'Selected from used logs' combo box. Click 'next' to enter the 'Log Similarity Aggregation' page.

    • In the 'Similarity aggregation' section select the log's columns you want to include and / or exclude in the similarity process. If no columns are selected, then the report runtime will include in the similarity computation all the log's fields which are not of type 'number' or 'date'. Enter in the 'similarity threshold' field the threshold above which log events will be treated as similar and aggregated into the same log event pattern.

    • In the 'Data Filters' section, select the filters that should be used is fetching the log records. Only log records that comply with these filters will take part in the report's computation.

    • In the 'Table display settings' you can set the number of results to be displayed in the that column section of the report runtime result. XpoLog collects only the top 5000 values with the occurrences of different

    • Click 'next' to complete the similarity's definition and return to the 'Similarity Aggregation Report' page.
    • In the 'Status' section, click on the check box to turn on the computation of the similarity's status. Select 'more than' or 'less than' and enter the number of records in the result to complete the definition of the similarity's status. If the checkbox is left unchecked, no status for that similarity will be computed.

  13. Repeat steps 8 and 12 to add new types of aggregations.

  14. After all the report's types have been defined, click 'save' to save the report definition.

 
Report monitors

Report monitors allow you to execute tasks based on the status of report runtimes. In order to set a report monitor, click the in the 'Report Definition' page on the 'monitor' link of that report. In the 'Report Monitor Settings' page, enter the name of the report monitor. In the Actions section you select of actions to take place both upon monitor success and monitor failure.