Graph: Visualization & Analysis of Azerbaijani Web

Abstract

An in-depth comprehension of the local Azerbaijani internet landscape is essential for examining the patterns of information dissemination and the impact on the local network, as well as assessing Azerbaijan's reliance on foreign sources during cyber-attacks or national crises. In order to enhance this understanding and optimize the acquisition of local data, it is essential to use a web crawler followed by graphical analysis. The objective of this project is to construct a comprehensive network diagram of the Azerbaijani web, and thereafter examine its interconnections and identify the most significant nodes. The objective of this project is to establish a comprehensive list of local websites, design a web crawler to go through each webpage and its external linkages, build a visualization using a graph-based approach to display important information, and use a ranking algorithm to quantify the influence scores. A Python crawler program is created to systematically search and get data from the database of local websites provided by the Ministry of Communication & Information Technologies. The extracted data is saved in both a Postgres database and a Neo4j graph database. The graph is composed of nodes that represent web sites, and edges that reflect relationships between them. A method for page ranking is used to quantify the significance of nodes. The findings mostly focus on visualisation, while some analytical tasks need separate implementation.