Release History and New Features

We are proud to unveil … the “Veiled Chameleon” (2024-12), the latest release of the webLyzard Web intelligence platform. Among its many new features, it allows you to manage keywords and export data more conveniently, and to share complete dashboard snapshots with colleagues through a simple permalink.

The release history below provides a quick overview of these features. We invite you to try them out by accessing SDG Intelligence dashboard developed for the United Nations, or to take a look at the official dashboard documentation for a detailed description of our data and visualization services.

2024-12 (Veiled Chameleon)

Dashboard Features

• Keyword Management. Select and filter keyword types to display as labels (concepts, persons, organizations, or locations), individually or in combination. This affects the tag cloud, trend chart, story graph and various other modules.
• Dashboard Snapshots. Share complete dashboard snapshots with others using a simple permalink that captures the current state and all your settings, including selected sources, time range, topics, and more.
• Data Export. Enhanced user experience by separating exports from the current session, allowing multiple requests to be processed simultaneously. Once complete, PDF and data exports will be accessible as download links in the header information bar.
• SKB Tooltip. Access in-depth information about knowledge graph entities with enhanced tooltips and better thumbnail displays. An additional tab in the tooltip features a description and a link to the external entity source.

Visual Analytics

• Story Graph. More colorful and flexible story graph by calculating stories for each selected topic or classification separately, and then assigning colors accordingly.
• Trend Chart. To create a more vibrant and versatile story graph, the new module calculates stories for each chosen topic or classification separately and then assign colors accordingly.
• Geographic Map. An additional option to display source locations at the country level (default representation: polar area chart), or target locations at the country, city, or street level (default representation: bubble chart).
• Rendering Improvements include the relation tracker with a larger number of entities in the radial layout, a faster pattern-matching algorithm to highlight search terms and usability enhancements across various other modules.

Dashboard Architecture

• Dashboard LITE. (1) A new drop-down menu has been added to switch between metadata categories such as sources, sentiment, recency, etc.; (2) autocomplete functionality to enhance search and filtering capabilities; (3) visualization labels can now be filtered by entity type (concept, person, organization, location).
• Dashboard Configuration. A collapsible sidebar enables users to minimize individual sidebar sections instead of hiding them, providing additional space for other sections that are more useful for the current task.
• Elasticsearch Cluster. Additional server nodes to improve the platform’s query performance, availability, and stability during concurrent or complex requests.
• Thumbnail Repository. Updated repository to better distinguish the various entity types, including persons, organizations, and locations.

Data and Content Services

• PDF Reports. Added a story list representation to the 16:9 slide report type and created more consistent legends across the various reports.
• Email Alerts. To enhance user experience and effectively differentiate multiple alerts associated with the same bookmark, each alert can now be assigned a descriptive label.
• Multi-Lingual Knowledge Extraction. Ongoing refinements to improve precision and recall of knowledge extraction algorithms, which are now available in nine different languages: Czech, Dutch, English, French, German, Italian, Portuguese, Slovenian, and Spanish.
• Embeddings. Dense vector embeddings to query for document similarity, with support for third-party embeddings to enable cross-modal searches in image and video content.
• Environmental Indicators. Ingestion of real-time weather data, including temperature, humidity, wind speed, and precipitation, and integration into the trend chart and geographic map.

Source Management

• Stakeholder Identification. Wikidata categories and longitudinal analyses of third-party Web indices to extract domain-specific URL and metadata collections. These collections are then used to include new Websites in the content acquisition.
• User-Generated Content. Addition of Bluesky content and more targeted capturing of YouTube content based on video categories.
• Source Metrics. Additional source-level attributes such as credibility, trust and neutrality, complement the existing reach and impact values.

2023-03 (Uzzell’s Lizard)

Dashboard Architecture

• Dashboard LITE. New user interface and default landing page, complementing the PRO version of the dashboard (see introductory video tutorial). Following a responsive design approach, it supports mobile clients and provides a more linear, simplified user experience.
• Performance Gains. (i) Server-side HTTP/2 for a higher number of parallel dashboard requests, significantly improving the response times of most components; (ii) extended application server cluster; (iii) refined deployment process for increased reliability and fault tolerance.

Dashboard Features

• Interactive Tooltips. Extended tooltips show additional keywords and a preview of top-matching documents to gain more insights on selected items – e.g., trend chart peaks, named entities, geographic bubbles, etc.
• Search Match Highlighting. (i) Classification. Instead of a uniform highlight color, the corresponding bookmark’s color is used to represent full-text matches; (ii) Speed of processing complex queries has been dramatically improved; (iii) Consistency of highlighting named entities across the dashboard, PDF reports, and email alerts.
• Keyword Associations. New lemmatization service to merge related references (singular vs. plural, spelling variations, etc.) into a uniform keyword label. When using the label to trigger a search, the query is automatically expanded to encompass all variations.
• Bookmark Management. (i) Option to add a descriptive comment to bookmarks, which is displayed on hover in the sidebar; (ii) faster sidebar updates when changing the data range or source selection, particularly in the case of longer bookmark lists.
• Emotion Detection. Following recent advances in affective knowledge extraction, the corresponding emotion categories can now be used in advanced search queries. The metadata view uses color coding to show and classify the individual occurrences of emotion terms.

Visual Analytics

• Tag Cloud. (i) Slider element to control granularity, i.e. the number of terms displayed in the tag cloud. (ii) Improved representation. To avoid ambiguity when displaying bi- and tri-grams, they are now separated with a “mid-dot” instead of a space.
• Trend Chart. (i) Peaks in line charts are now automatically labeled with the most prominent keywords; (ii) significant performance improvements for queries that span multiple years or many sources and languages.
• Geographic Map. Extended settings menu and new “polar area” representation of data points, as an alternative to the circular “bubbles” for improved visual comparison of multiple datasets.
• Keyword Graph. Improved center-node handling to decrease overlap.

Data Services

• PDF Report Generator. (i) Redesigned layout to show the number of matches for each query and adapt the color-coding visualizations accordingly; (ii) Peak labels for trend charts; (iii) download icon in full-text mode to export single documents.
• Named Entity Extraction. More granular classification of named entities by occupation (e.g. politicians, journalists, etc.), geographic scope (countries, cities, etc.), or industry code (according to the NACE standard).
• Cross-Lingual Support. Extended content processing pipeline to support the analysis and enrichment of Portuguese and Italian content.
• Text Classification. Deep learning-based text classification component, currently trained to detect IPTC news genres for English and German content.

2021-06 (Thorny Devil)

Visual Tools

• Geographic Map. (i) Improved granularity and accuracy of displayed bubbles, including street-level geotags for selected regions; (ii) rendering of multiple bubbles per document, according to extracted geographic entities; (iii) revised configuration menu for specifying the number of displayed bubbles and trajectories; (iv) improved rendering of overlapping bubbles; (v) transparency for non-dominant topics.
• WYSDOM Success Metric. Multiple checkbox selection among all bookmarks, associations and metadata categories. This eliminates the need for a mode switch and the restriction to sentiment and desired vs. undesired associations. Bipolar slider widgets to place a selected time series above or below the axis.
• Word Tree. (i) Extended display to provide a longer list of full-text matches selectable as the root node; (ii) new optimization option to exclude stop words and improve sentence clustering.
• Cluster Map. (i) Improved initial animation, (ii) extended tooltip and label display; (iii) revised placement algorithm to exclude small clusters containing fewer than three nodes.

Dashboard Header

• Context Filter. The previous “Advanced Search” menu option has been replaced by a gear icon located within the text input box to access the context filter settings.
• Keyword Filter. New menu option to avoid specific keywords and entities in the list of associations and as visualization labels. There are two stop list columns – the list on the left only applies to the current session, while the terms on the right are stored permanently.
• Temporal Animation. Controls to set the duration of an animation, the size of incremental steps and the average window length. Synchronized animation capabilities for the trend chart, geographic map, entity map and tag cloud.

Search Results

• Document Management. (i) User rating system to assign between one and five stars to a documents, accessible via a corresponding metadata sidebar category; (ii) shopping cart functionality to manually select a collection of documents to be exported; (iii) full-text document preview in the right sidebar.
• Boost and Demote Terms. Accessible via the floating menu of the content area, two new options allow specifying two different term lists to modify the relevance ranking: documents containing a boost/demote term will be shown at the beginning/end of the results list.
• Query Highlighting. (i) Improved highlighting of prefix and postfix matches; (ii) correct processing of query term matches in document titles; (iii) support of Unicode characters and (iv) improved handling of word boundaries.
• Similar-To Queries. Embedded tag cloud, geographic map, keyword graph and story graph based on the similarity of content vis-à-vis a supplied text document.
• Data Export. (i) Export files as email attachments, in addition to interactively downloading them via the Web browser; (ii) option to include the full text of documents in the PDF search results export.

Dashboard Architecture

• Query Engine. Update to Elasticsearch 7.13 for improved performance and stability, new indexing and query strategies for geographic data, improved time- and term-based aggregations as well as faster recovery in case of node failure.
• Mobile Dashboard. Complete rewrite of the mobile application using Vue.js 3, with the option to embed the entire application including context filter and tabs to switch between different visualizations.
• Sidebar Visualizations. (i) Placement of settings menus in the upper left corner; (ii) improved placeholders for the initial rendering or in case of missing data to better reflect expected final appearance.
• Multilingual Features. (i) Extend processing pipeline to provide content from Italian sources; (ii) add support of Dutch interface language; (iii) improve user interface localization by switching to gettext utilities.
• Login and Session Control. (i) Revised user interface and add support of external authentication sources, e.g. SAML2 (Active Directory) and OAuth2; (ii) automatic creation of users and synchronization of access rights based on user roles; (iii) keep-alive health checks to the server to detect interruptions of a session or networking issues.
• Technology Stack. (i) Vue.js: Rewrite user interface code to migrate to the Vue.js 3 framework, resulting in improved maintainability and reduced code complexity; (ii) Webpack 5 JavaScript build framework for future extensions and bundle size improvements.

2020-06 (Sagebrush Lizard)

Visual Analytics

• Temporal Animation. Extension of the geographic map and charting library to show the evolution of the public debate over time – using sliding average aggregations, accessed via new animation controls in the timeline date picker.
• Geographic Map. Pan and zoom operations dynamically update the map, providing more details without cluttering the display at lower zoom levels.
• Scatterplot. Major revision to render different data types (topics or metadata attributes) along both axes and the bubbles, replacing the standard frequency vs. sentiment display of the previous version.
• Trend Chart. (i) New time series “Impact”, based on frequency but considering the reach of the document’s source; (ii) Complementing the running average, the deviation from the average base value over the past months period allows to compare the relative importance of topics with diverging absolute frequency.
• Story Graph. Performance gains, advances in clustering and label post-processing to better align results with the Associations sidebar, thumbnails for all Web sources.
• Cluster Map. Option to select the number of documents shown and an improved rendering algorithm to better adjust the map to windows with different aspect ratios.
• Keyword Graph. More precise user control when dragging individual nodes.
• Layout. Visual appearance adjustments and new highlighting procedures for charts and various visual widgets.

Dashboard Features

• Metadata Sidebar. New attributes including Emotions (according to Plutchik’s Wheel of Emotions), Recency (based on temporal classification into four quartiles) and Top Sources, leveraging the color-coding capabilities of the Topic Comparison mode.
• Adaptive Tooltip. Horizontal bar chart to show the distribution of selected topics and metadata attributes together with a multi-color line chart to show their development over time.
• Metadata Aggregation. Modified searches and filter operations that give preference to the more accurate sentence-level aggregations over document-level aggregations whenever possible.
• Disambiguation. Phrase editor option to specify how many regular expression terms need to match for inclusion into the set of search results. This can improve the precision of the query at the cost of lower recall, especially for terms that are ambiguous without additional context information.
• Negation Detection. (i) Standard prefixes, grouped by part of speech, to support on-the-fly negation detection; (ii) custom prefix processing for additional user control.
• Entity List. Sorting persons, organizations and locations triggers a remote query to retrieve global results, as compared to just re-sorting the already retrieved list.
• Advanced Search. (i) New field type to allow searching in “Text or Title” without having to use a Boolean OR; (ii) option to search for multiple terms appearing in the same sentence; (iii) validity check for source identifiers.
• Timeline Datepicker. Refactored to allow a more precise date range setting and fix a daylight savings time issue.
• Index Structure. Update of the indexer and advanced search to capture (and query for) the length of a document in terms of contained characters, words and sentences.
• Performance Optimizations. Updating widgets to version 5.x of D3.js and reducing bundle size by removing superfluous CSS definitions improved download speed and browser performance.

WYSDOM Success Metric

• Settings Menu. Dedicated sidebar that opens automatically on activation, containing (i) sliders to set the relative weights of the indicators, (i) actions to reset the weights and show/hide the overall WYSDOM score, and (iii) the selection of desired and undesired topics.
• Temporal Granularity. Use the current chart settings (daily vs. hourly intervals) for WYSDOM queries and visualizations.
• Statistical Indicators. If there are indicators associated with the selected bookmark (e.g. audience metrics such as visitors and page views), these indicators can be accessed and adjusted using separate slider elements.
• Standalone Version for embedding the chart in third-party applications, with a new color scheme to better highlight desired and undesired associations.

Data Services

• Knowledge Graph. (i) Integration of awareness days, (ii) endpoint and documentation for the entity type “events”; (iii) endpoint to add custom regular expressions for an entity, stored in the triple store, (iv) cleanup of obsolete data, (v) migration of the cache to Elasticsearch 7.x for increased performance.
• Sentiment Analysis. (i) Revision of the unicode emoji lexicon; (ii) sentiment annotation service for Dutch content, initial version that is based on an adjective-only sentiment lexicon.
• Geotagger. Unique street names for cities with a population of 50,000 or more within the DACH region, including alternate names, parent city and coordinates.
• Report Generator. Download dropdown option to select either sentiment or content source for color-coding the results.
• Synonym API. Synonym and antonym suggestions, including retrieval of inflected versions and filtering by part of speech.

2019-06 (Red Iguana)

Dashboard Features

• Search Results Selector. Redesigned hierarchical layout with five main categories – Documents, Sentences, Sources, Entities and Relations. Typically, each of these categories contains at least one list view and one visualization (e.g. word tree for sentences or scatter plots for entities and sources).
• Info Bar. Context-aware help text feature as part of the portal header, including a drop-down menu to replace the help text with the most frequent sources or references to named entities (persons, organizations, locations).
• Left Sidebar. Look and feel aligned with the visual tools of the right sidebar; support of multiple categories as well as drag and drop operations; revised metadata category, including source country as an additional attribute.
• Source List. Replaced frequency by impact as the default sorting attribute to highlight the most influential sources.
• Threaded View. Support for viewing and navigating hierarchical conversation threads and document-to-document relations.
• Keyword Blacklist. New configuration menu option to specify terms not to be considered in the list of associations and the various keyword-based visualizations (complementing the server-side blacklist module)
• Performance. Reduced initial load time by removing dependencies and separating optional components – e.g. WYSDOM and radar chart.

Visual Analytics

• Multi-Color Mode. Alternative color scheme based on the selected topics or metadata attributes, visible in the portal mode “Comparison”; including stacked horizontal bar chart to reflect result composition in the source and entity lists.
• Keyword Graph. Pre-fetching required data and calculating the entire graph prior to force-directed rendering leads to a more stable and visually appealing layout in both the dashboard and the PDF version.
• Geographic Map. Support of polygon data, extended dashboard controls, improved vector tiles and refactoring of the entire module.
• Trend Charts. Data granularity (days, minutes, hours) as new floating menu options, including “automated” that adapts to the selected time interval.
• Legends. Implementation of customizable legends for all bubble-based visualizations (geographic map, scatterplot, keyword graph).
• Tag Cloud. Improved tag distribution and zoom behavior.
• Story Graph. Automated sorting to improve the ranking of documents within a given story; increased performance and minor bug fixes.
• Image Server. Added support for favicons in multiple resolutions.

PDF Report Generator

• Knowledge Graph Integration. Provide entity descriptions and show thumbnails for opinion leaders as well as flags for countries (SVG as the preferred format, with PNG fallback).
• 16:9 Slides. Complementing the A4 reports optimized for high-quality print output, this new module generates customizable slide decks in 16:9 format.
• Story Detection. Embedded document links provide direct access to the original content.

Data Services

• Content Acquisition. Customizable content classification at mirroring time for hybrid Web sources (news, jobs, real estate, etc.); iCal event support and structured forum extraction; GeoJSON filter for CKAN open data integration.
• Sentiment Analysis. Improved n-gram support and named entity processing; better handling of apostrophes and other special characters; preservation of text-based emoticons during the tokenization process; extended set of negation triggers.
• Named Entity Recognition. More granular geotagger based on a new disambiguation algorithm that identifies and confirms unique, non-ambiguous metadata items in the knowledge graph.
• Term Cache. High-performance caching mechanism to speed up internal operations and support third-party applications.
• Reach Metrics. Extended and updated set of metrics (that will become a global dashboard filter); refined normalization across channels.
• API Framework. Provision of new REST APIs (Story API, Keyword API, Event API) for both internal services and third-party applications.

2018-12 (Queensland Lizard)

Dashboard Features

• Header Design. A set of overlays accessible via the new header replaces various interactions previously distributed across different icons and drop-down menus. The overlays include: Configuration, Data Export, Advanced Search, Date Range and Data Sources.
• Configuration Overlay. Dialog to select (i) the portal mode, which determines whether the total number of references or just co-occurrences with the current search term are used for presenting results, (ii) the sidebar elements and widgets to be shown, and (iii) the interface language.
• WYSDOM Success Metric. In addition to the global default, users can now store desired and undesired settings together with each topic.
• Video Integration. Enhanced compatibility with multiple social media channels, including embedded playback and thumbnail display in the Story View.

Visual Analytics

• Story Detection. Automated clustering of related documents (= stories) in digital content streams, represented as a streamgraph and corresponding table (including a lead article for each story and a list of related documents).
• Multi-Color Mode. Extension of the geographic map, tag cloud, keyword graph and cluster map modules to support multiple colors for representing bookmarks, associations or metadata elements selected in the left sidebar.
• Geographic Map. Rendering of Scalable Vector Graphics (SVG) tiles instead of bitmap tiles, including better map boundary handling.
• Cluster Map. Revised positioning algorithm to make better use of the available space, especially along the horizontal axis.

Data Services and Infrastructure

• PDF Reporting. Story Detection report including streamgraph and overview of top stories; export of multiple user-selected reports as an integrated PDF file; email alert extension to deliver weekly or monthly reports.
• Social Media Reach. Improved algorithm to compute per-source reach for various social media channels, based on the number of followers and likes.
• Sentiment Analysis. Bigram processing enabled for the negation detection module, avoiding that exceptions such as “not only” trigger a negation.
• Distributed Processing. Addition of three physical nodes to the frontend server cluster and upgrade to Elasticsearch 6.5.1.

2018-06 (Panther Chameleon)

PDF Reporting

• Opinion Leadership. Scatterplot and corresponding table to show how often opinion leaders were mentioned together with the search term, and whether this happened in a positive or negative context.
• Sentence Analysis. Color-coded word tree to convey major threads in the public debate, followed by tables with recent positive and negative mentions.
• Documentation. Optional appendix page with a summary of the available report types, including a description of their main features.

Visual Analytics

• Geographic Maps. Custom basemap for a more granular view of the regional distribution of search results, using an updated geographic map module for multiple basemaps.
• WYSDOM Chart. Improved ingestion mechanism for performance and audience metrics; removed dependencies of external libraries beyond D3.js.
• Word Tree. Slider interface element to control the level of detail and number of sentences shown; support of prefix and suffix “*” queries by fetching the most popular matching terms; e.g., eco* -> ecological, economy, etc.

Dashboard Design

• Incremental Load. Clearer indication of queries in progress by greying out previous datasets and transitioning into the new state once new results become available.
• Advanced Search. To simplify query definition and reduce the number of items, the text editor initially developed for managing regular expressions is now available for additional fields including URL, source identifier, target location and keywords.
• Date Format. Unified display of date references across all dashboard elements using “dd mmm” format.
• Initial Load. Reduced size of initial download including script bundles; more than 200 kb for the mobile dashboard, and 30 kb for the desktop version.

Knowledge Extraction

• Content Acquisition. Added support for the ingestion of Reddit content; dynamic mirroring component to collect external resources referenced via outgoing links, e.g. Youtube and Facebook videos.
• Microdata Enrichment. Extraction of Schema.org concepts such as event content and video content from HTML5 microdata, where available.
• Organizational Metadata. Capturing and enriching site owner metadata to improve audience metrics and user-generated content classifications.

2017-12 (Ocelot Gecko)

Visual Analytics

• WYSDOM Success Metric. Enhanced charting module with a dedicated settings dialog and the option to select desired and undesired topics from any of the defined categories.
• Bar Charts. (1) New horizontal bar chart as an aggregated alternative to the trend chart, particularly useful for less dynamic sources. (2) Vertical bar chart as a more information-rich alternative to the donut chart; (3) redesign of the chart settings dialog to align it with other visualizations.
• Sentiment Line Chart. Increased precision by rendering sentiment values on the sentence level instead of the broader document sentiment.
• Relation Tracker. Support of stacked relations, additional sorting options (entities by type, sources by reach and impact), and improved layout and interaction features.
• Word Tree. To avoid sparse word trees in the case of less frequent terms within a composite query, a revised content retrieval module provides an independent set of sentences for each matching term.

Dashboard Features

• Continuous Scrolling. For the document and sentence lists, continuous scrolling and dynamic content updates replace the manual paging through the result list in 50-document intervals.
• Initial Load. Improved user experience by immediately displaying placeholder elements and static elements such as chart grids and menus to inform the user about what will be shown in the given module after the data arrived.
• Topic Definition. In addition to creating new topics from scratch, users can now save topic definitions based on the currently active search.
• Wildcard Queries. For queries without term restrictions (“*”), the system hides both the sentence view and the word tree.
• Client Side Error Reporting. Central error tracking service to identify Web browser incompatibilities and client side errors, providing detailed insights for faster software updates and patches.
• Landing Page. Centered search box intended as a simplified landing page for information retrieval-focused applications (as compared to topic comparisons) in order to speed up the initial load and the first query.

Data Services

• Web Crawling Framework. Completely rewritten, horizontally scalable content acquisition component to shorten crawl intervals, increased recall and improved parsing of the latest Web frontend technologies.
• Document Relations. Increase versatility of the webLyzard document model to support the mapping of threaded dialogs and other dependencies among content items.
• Increased Coverage. Additional Web sources and social media search terms in multiple domains including politics, tourism and consumer brands.

2017-06 (Northern Alligator)

Data Export and Reporting Features

• Report Generator. Automated compilation of on-the-fly PDF summaries, for example to integrate analytic results into weekly management updates; currently supported report types are Trend Analysis, Topic Associations, Cross-Media Analysis, and Geographic Distribution.
• Export Menu. Instead of using a separate sidebar element, restructured and extended export options are now available via the standard header menu.
• E-Mail Alert. Following the removal of the right sidebar, the e-mail alert configuration has been transferred to the topic editor window.

Visual Analytics Dashboard

• Timeline Selector. Complementing the calendar-based date picker, the timeline provides a visual overview of the temporal search results distribution, including interactive elements to to select custom date ranges.
• Brand Reputation Radar. New algorithm to calculate associations that achieves a more balanced normalization across radar chart dimensions.
• Mobile Version. Redesigned landing page with a centrally located search field, optimized loading times and improved overall performance.

Data Services

• Content Acquisition. Hybrid access method that combines streaming and search APIs to improve recall and enable the ingestion of historic data.
• Named Entity Recognition. Hybrid profiles to dynamically adapt the granularity of the geotagging process and set the minimum size of a city depending on its proximity to a specified region of interest.

The earlier platform releases until March 2017 [A-M] are described in a separate technology archive document.