Lulai Zhu

Optimizing Quality-Aware Big Data Applications in the Cloud

Uncategorized Comments Off

Oct 262018

Many analysts point out that we are experiencing years in which technologies and methodologies that fall within the sphere of Big Data have swiftly pervaded and revolutionized many sectors of industry and economy, becoming one of the primary facilitators of competitiveness and innovation. IDC reported that the Big Data market will grow from $150.8 billion in 2017 to $210 billion in 2020, with a compound annual growth rate of 11.9%. Among the enabling technologies, Apache Hadoop was the first successful data-intensive framework, but its predominant market share is declining in favor of newer projects, such as Apache Spark.

TOSCA’s Important Role in the DICE EU H2020 Project

Uncategorized Comments Off

Oct 262018

Within the first open source framework for quality-aware DevOps for Big Data applications – DICE, TOSCA provides the perfect foundation for bridging the world of off-line application modelling and the running instances of this application. Its extensibility enabled automating deployments of complex data-intensive applications.

Deploying the DICE Simulation Tool in the News Orchestrator DIA

Uncategorized Comments Off

Jan 162018

Scalability, bottleneck detection and simulation/predictive analysis are some of the core requirements for the News Orchestrator DIA. The DICE Simulation tool promises that it can perform a performance assessment of a Storm based DIA that would allow the prediction of the behaviour of the system prior to the deployment on a production cloud environment. The News Orchestrator engineers are often spending much time and effort in order to configure and adapt the topology configuration according to the target runtime execution context. Introducing a tool that can perform such a demanding task efficiently would clearly increase the developer’s productivity and also facilitate their testing needs.

DevOps: A Quality Assessment Experience

Uncategorized Comments Off

Dec 132017

In a previous article of this Blog, we discussed the importance of assessing quality during the development of data intensive applications (DIA). In particular, we explored the performance and reliability properties of DIA and presented a Simulation tool (SimTool) that helps on this purpose. This article extends such contribution, concretely for addressing the quality topic in the DevOps context. The core idea of DevOps is to foster a close cooperation between the Dev and Ops teams. Probably, the reader will also be interested on taking a look of what the DICE project proposes at this regard.

Detecting Anomalies during App Development

Uncategorized Comments Off

Oct 312017

During the development phase of a Data Intensive Application (DIA) using Big data frameworks (such as Storm, Spark, etc.) developers have to contend with not only developing their application but also with the underlying platforms. During the initial stages of development bugs and performance issues are almost unavoidable and most of the time hard to debug using only the monitoring data. The anomaly detection platform is geared towards automatically checking for performance related contextual anomalies.

5 Reasons to Use Fault Injection in DevOps

Uncategorized Comments Off

Oct 312017

Bringing development and IT operations together can help address many application deployment challenges. To address areas of quality these challenges require a toolset to manage and measure performance and improve reliability. There is a need for not only resilient platforms but also robustness of the data intensive applications that run inside them.

A Fault Inject Tool (FIT) is part of that solution and one way to provoke scenarios which will emphasise the impact of sometimes otherwise hidden issues. The FIT enables controlled causing of cloud platform issues such as resource stress and service or VM outages, the purpose being to observe the subsequent effect on deployed applications.

The FIT is being designed for use in a DevOps workflow for tighter correlation between application design and cloud operation, although not limited to this usage, and helps improve resiliency for data intensive applications by bringing together fault tolerance, stress testing and benchmarking in a single tool. Here are 5 compelling reasons why a FIT tool is useful for the developers of data intensive applications.

JMT Petri Net Extension for Performance Analysis of Big Data Applications

Uncategorized Comments Off

Sep 102017

JMT (Java Modelling Tools) is an integrated environment for performance evaluation, capacity planning and workload characterization of computer and communication systems [1]. A number of cutting-edge algorithms are available for exact, approximate and asymptotic analysis of queueing networks (QNs), with either product-form or non-product-form solutions. Users can define and solve models through a well-designed graphical interface, or optionally an alphanumeric wizard. Released under GPLv2, JMT benefits a large community of thousands of students, researchers and practitioners, with more than 5,000 downloads per year.

“Gazing” the Clouds: Cloud Applications Monitoring, and what’s going on in industry…

Uncategorized Comments Off

Sep 042017

The advent of cloud computing triggered a huge change in software release cycles for an increasing number of companies embracing cloud technologies as the 21st century’s technological utility… Where once your company invested in large, upfront investments in physical servers, that same strategy is increasingly being replaced by on-demand and pay-per-use cloud access – at the same time, complex manual deployment procedures are increasingly being automated in the context of DevOps and connected technologies… What is the organizational and technical consequence of these phenomena?

Release 0.3.4 of DICE Deployment Service

Uncategorized Comments Off

Jun 192017

We are happy to announce the release 0.3.4 of our DICE Deployment Service and version 0.7.0 of the DICE TOSCA technology library. With these components, we aim to remove one big hurdle on the path to the world of Big Data: setting the components up and wiring them to have all the parts play along nicely. We also want to enable the users to easily run their application in a number of private and public clouds without any worry of being locked into a particular one. This release introduces a unified approach to deploying blueprints to OpenStack, Amazon EC2 or Flexiant Cloud Orchestrator without needing to change anything in the blueprint.