Osint Analytics is a Norway-based big data analytics company that serves businesses with their custom-developed OSINT solution.
The product is a data mining and analysis tool that allows our client to effectively retrieve, systemize and analyse information from publicly available online sources.
Osint Analytics partnered with Caddiesoft to extend their tech team with our full-stack PHP developers to build and support the client tool, Tilskuddsportalen database, as well as Osint Analytics’ own website.
of partnership with Caddiesoft
When Osint Analytics partnered with Caddiesoft back in 2016, there already was a functioning solution developed and released by the client. The initial request was to support the existing version of the product, as well as to fix emerging errors and extend functionality.
However, the legacy system proved to be underperforming due to such factors as outdated technologies and overstructured approach to the development process. The need for a brand new solution arose, which was to replace the existing one and allow the client to provide their services in a more streamlined and cost-effective way.
After thorough review of an existing system and scrupulous analysis of the client’s business requirements, our engineers developed a brand new product using advanced tools and thought-out strategies.
It then replaced the legacy project and allowed our client to expand the span of their services with additional solutions.
Products we built
A database portal allowing Norwegian organizations, local councils and charities to find and apply for grants offered by the government.
A product developed to help customers access information relevant to their business by means of setting specific search parameters with the crawler.
An automated solution for carving out and systemizing data collected by OSINT technology.
An interface allowing users to access scientific articles and literature across university libraries with the help of OSINT technology.
Challenges and solutions
Outdated and underperforming legacy system
The main issues that our developers faced with the legacy system were unnecessary dependencies in the code, the lack of comprehensive documentation, extra pieces of code in the product, excess load on systems, and user data security threats.
After thorough system analysis and discovery of the bottlenecks, it was decided to address those issues in two stages.
- Support the existing product, improve the feature set, and find possible solutions considering all the limitations of the legacy system, all with minimal downtime.
- Develop a brand new system with optimized architecture and design, using the latest technologies and taking into consideration business needs and requirements.
Having analysed MongoDB, our engineers discovered that the database was heavily overloaded with big amounts of obsolete user data.
- We developed automated scripts that would promptly delete all the user information that would become outdated.
A careful evaluation of the MySQL database revealed abundant slow queries in the system. These were the obvious reasons for a poor and slow performance.
- Our developers optimized the database by creating indexes based on the list of slow queries.
System performance and security
The outdated tools that the product was built with resulted in overall low performance, numerous latency issues, and potential security breaches.
- The use of PHP 7.1 instead of the 5.2 version, as well as the Zend framework, allowed us to increase the speed and improve the security of the system.
- In addition, we incorporated Elasticsearch into our development process to improve the overall performance of the search engine.
Maintenance quality of Tilskuddsportalen
The initial design of the legacy system also introduced certain difficulties with support and maintenance.
A good instance is the original structure of clients’ profiles in Tilskuddsportalen, which were organized as separate projects on one server. This structure resulted in local errors and consecutive long update time (from 3 to 4 hours).
- Our engineers organized all the profiles in one project. Since the issue was of the architecture level, the complete redevelopment took place. This significantly decreased the update time from hours to approximately 20 seconds.
Maintenance quality of Dr. Tide project
With Dr. Tide, on the other hand, each client’s profile is mounted on the individual server, due to the business concept of the product (premium and customizable memberships).
This organizational model didn’t allow us to move all the profiles to one server, which, similar to Tilskuddsportalen, drastically increased the update and maintenance time, which was to be optimized somehow.
- Our team used the Ansible tool and wrote cookbooks for it to optimize and streamline setup and maintenance processes.
Caddiesoft serves Osint Analytics with our two full-stack engineers that work from our development centre in Ukraine.
Full-stack PHP software developer
Full-stack PHP software developer
Working on the projects for Osint Analytics allowed me to learn new ways of organizing the working process in a more effective and streamlined way. I am glad that I am a part of the Osint Analytics team, which gives me the opportunity to learn and apply modern technologies and innovative tools.Vladimir
Full stack PHP developer
Tell us about your project
Let us know what your project is about, and we will contact you to discuss requirements and other details.