Skip to main content

Project Introduction

Notification System for GraphSpace


GraphSpace is an user-friendly web-based platform that collaborating research groups can utilize for storage, interaction, and network sharing. A GraphSpace user can import graphs created in Cytoscape, upload them through a REST API, interact with them by customizing and saving layouts, share them within and between groups of collaborators, search for different graphs, and organize them using tags. These functions are enabled through GraphSpace’s comprehensive REST API, which allows users to communicate programmatically.

Notifications are integral to all web applications as they are the means to tracking all important information. It is most optimal when a notification system is decoupled from the main application flow, because we can have a clear distinction between the utilities of the two applications. In this project, there are three primary types of notifications that need to be implemented: Owner, Group and Watching. A owner notification is defined by one where users automatically receive a notification when they create a resource (graph, layout or group) on GraphSpace. Group notifications, on the other hand, will automatically be received by users when they join a group or when members of group that they are part of shared resources. Lastly, Watching notifications will be automatically received by users when they are actively interacting with a graph or layout, but aren't necessarily the owner of it; users will receive this type of notification only if they have manually signed up for the Watching notification.

So, to implement this notification system, I will be using the messaging queue system, Apache Kafka. While I considered RabbitMQ as an alternative service, its license Mozilla Public License (MPL) is incompatible with GraphSpace’s General Public License (GPL) v3. As this system should be decoupled from the main application flow, the main application will act like the producer, creating desired notifications and adding those to the messaging queue (see Figure 1). The producer will send different types of notification messages under different topics, and the consumer will add these notifications along with the status and type (from the topic of the message) to the same PostgreSQL database used by GraphSpace. Another component of the project will be implementing sockets in Django. This will be done using the django-socketio package. A socket is one endpoint of a two-way communication link between two programs running on the network. It allows real time, bidirectional communication between the web client and server. This allows for logged in users to see notifications in real time without refreshing the page. The notification will be pop down from the top right corner of the webpage the user is currently in.


A scheduled task, implemented using django-cron will email all unread notifications which have not been emailed before, to all users. Django-cron will allow us to write cron logic in Python to schedule tasks. The tasks will be scheduled to run whenever the GraphSpace server has less load. I will decide the criteria after discussion with my mentors and the GraphSpace team. The emails will be sent to only those users who have opted in for this service. The notification that have been emailed will be marked so in the database, so that they are not sent again.

In addition to adding a notification button or icon to the user’s dashboard displaying the number of unread notifications, a webpage to show all notifications will also be created. A user can then mark the notification as “read” or browse all notifications based on their read state, content, and type; this information can be accessed by clicking on the notification icon. The notification will be populated on this page by an API call to the notification application with request attributes. These request attributes will act like filters to get notification of a specific state, content and type.

Disclaimer: The implementation may change as I explore on how to build this system better.
Labels:

Comments

  1. tavisqualmanMarch 3, 2022 at 2:28 PM

    Casino, Hotel and Racetrack, Las Vegas - Mapyro
    Find your way around the 창원 출장안마 casino, find where 세종특별자치 출장샵 everything 화성 출장마사지 is located, and learn 평택 출장안마 about the casino's 포천 출장마사지 amenities and activities.

    ReplyDelete

Post a Comment

Popular posts from this blog

Django + Kafka

Q & A of connecting Apache Kafka and Django Problem #1 Kafka consumer should always be listening for new messages in the queue. The consumer should be running in parallel to GraphSpace app (producer). Solution There are multiple ways to do this. We create another Django project say called "GraphSpace notification consumer" which starts along the GraphSpace application and establishes a connection with Kafka. This is an overkill for a simple consumer. We can use celery for multi-threading. Celery itself uses Redis or RabbitMQ as a queue for tasks. This setup might work for a large scale multi-threading system but for a simple setup of running a consumer this is a overkill. We will be running 2 queues (Redis and Kafka), one for queuing task and another for queuing content. I tried this setup with the following architecture: I will recommend keeping the architecture simple, run the consumer code on a different thread in daemon mode. Problem #2 Where
2 comments
Read more

Websockets in Django

Q & A What are websocket? Why do we need it? Quoting from wikipedia , "WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API in Web IDL is being standardized by the W3C." WebSockets are used for streaming messages. To implement real-time notification we need to stream messages from the server to the client without a refresh or making an HTTP request from the client. Hence websockets. How do we implement websockets in django? We will be using django-channels  (channels) for websockets. Django, by default does not support websocket. Channels is a django project which allows Django to handle websockets, HTTP and HTTP2 requests. But how does channels implement websockets in WSGI server (gunicorn) which does not support websockets. Simple we don't. Instead, we will be using  Daphne , an interface serv
3 comments
Read more

Kafka Clients 101

In this post, we will look at packages to build a real time notification system for GraphSpace. GraphSpace is built in Django. The requirements for our system are Decoupled from the main app ,i.e, should be non-blocking Use Apache Kafka as the message queue system Follow a producer-consumer architecture with broker, Kafka Able to handle different types of notification: Group, Owner and Watching The above requirements can be satisfied if we have a package that can connect to Kafka, have an ability to handle different types of notification and is non-blocking. To handle different type of notifications we can have producers and consumers send and receive messages with different topics. So the package we need should be able to dynamically do so. Instead of having say 10 producers for 10 topics and establishing 10 connections to Kafka, it is better we have 1 producer which can send messages with 10 different topics. Similarly 1 consumer should be able to subscribe to 10 different
5 comments
Read more