Osint Analytics is a Norway-based big data analytics company that serves businesses with their custom-developed OSINT solution.
The product is a data mining and analysis tool that allows our client to effectively retrieve, systemize and analyse information from publicly available online sources.
Osint Analytics partnered with Caddiesoft to extend their tech team with our full-stack PHP developers to build and support the client tool, Tilskuddsportalen database, as well as Osint Analytics’ own website.
6+ years
of partnership with Caddiesoft
company
Dedicated development team
6 years, ongoing
Background
When Osint Analytics partnered with Caddiesoft back in 2016, there already was a functioning solution developed and released by the client. The initial request was to support the existing version of the product, as well as to fix emerging errors and extend functionality.
However, the legacy system proved to be underperforming due to such factors as outdated technologies and overstructured approach to the development process. The need for a brand new solution arose, which was to replace the existing one and allow the client to provide their services in a more streamlined and cost-effective way.
After thorough review of an existing system and scrupulous analysis of the client’s business requirements, our engineers developed a brand new product using advanced tools and thought-out strategies.
It then replaced the legacy project and allowed our client to expand the span of their services with additional solutions.
Products we built
Tilskuddsportalen
A database portal allowing Norwegian organizations, local councils and charities to find and apply for grants offered by the government.
Dr. Tide
A product developed to help customers access information relevant to their business by means of setting specific search parameters with the crawler.
Client tool
An automated solution for carving out and systemizing data collected by OSINT technology.
Trap
An interface allowing users to access scientific articles and literature across university libraries with the help of OSINT technology.
Challenges and solutions
Outdated and underperforming legacy system
The main issues that our developers faced with the legacy system were unnecessary dependencies in the code, the lack of comprehensive documentation, extra pieces of code in the product, excess load on systems, and user data security threats.
After thorough system analysis and discovery of the bottlenecks, it was decided to address those issues in two stages.
- Support the existing product, improve the feature set, and find possible solutions considering all the limitations of the legacy system, all with minimal downtime.
- Develop a brand new system with optimized architecture and design, using the latest technologies and taking into consideration business needs and requirements.
Overloaded MongoDB
Having analysed MongoDB, our engineers discovered that the database was heavily overloaded with big amounts of obsolete user data.
Solution
- We developed automated scripts that would promptly delete all the user information that would become outdated.
Slow queries
A careful evaluation of the MySQL database revealed abundant slow queries in the system. These were the obvious reasons for a poor and slow performance.
Solution
- Our developers optimized the database by creating indexes based on the list of slow queries.
System performance and security
The outdated tools that the product was built with resulted in overall low performance, numerous latency issues, and potential security breaches.
Solutions
- The use of PHP 7.1 instead of the 5.2 version, as well as the Zend framework, allowed us to increase the speed and improve the security of the system.
- In addition, we incorporated Elasticsearch into our development process to improve the overall performance of the search engine.
Maintenance quality of Tilskuddsportalen
The initial design of the legacy system also introduced certain difficulties with support and maintenance.
A good instance is the original structure of clients’ profiles in Tilskuddsportalen, which were organized as separate projects on one server. This structure resulted in local errors and consecutive long update time (from 3 to 4 hours).
Solutions
- Our engineers organized all the profiles in one project. Since the issue was of the architecture level, the complete redevelopment took place. This significantly decreased the update time from hours to approximately 20 seconds.
Maintenance quality of Dr. Tide project
With Dr. Tide, on the other hand, each client’s profile is mounted on the individual server, due to the business concept of the product (premium and customizable memberships).
This organizational model didn’t allow us to move all the profiles to one server, which, similar to Tilskuddsportalen, drastically increased the update and maintenance time, which was to be optimized somehow.
Solution
- Our team used the Ansible tool and wrote cookbooks for it to optimize and streamline setup and maintenance processes.
Tech stack
PHP
Yii2
Zend
Vue.js
MS SQL
MongoDB
Redis
Elasticsearch
Docker
Team
Caddiesoft serves Osint Analytics with our two full-stack engineers that work from our development centre in Ukraine.
Vladimir Boiko
Full-stack PHP software developer
Yaroslav Chesnokov
Full-stack PHP software developer
Testimonials
Tell us about your project
Let us know what your project is about, and we will contact you to discuss requirements and other details.