bing bong: An Rss robot compatible with QQ.

title: bing bong: An Rss Robot Adapted for QQ
date: 2021-08-27 22:16:04
tags:

Long time no see!

It has been almost half a year since the last blog post. In the past six months, I have been busy with schoolwork and then with work, which has kept me from having time to write blog posts. Fortunately, with the start of the new semester (and leaving my job), I finally have some free time to write an article!~

This article mainly discusses the program structure of bing-bong, which is a QQ robot for subscribing to Rss feeds. As friends in the Linux community should know, I recently wrote a QQ robot for subscribing to Rss feeds (how do you know? Of course, it's because the Linux Tea Party is the official designated trial group for bing-bong!). Due to time constraints, it didn't turn out as perfect as I imagined. Instead, it has various flaws in my opinion. Whether it's to allow everyone to participate in the modification or to clarify my thoughts for future refactoring, it is necessary to reorganize the program logic.

Source

As usual, let's first introduce the source of the project. Since there is a similar introduction in the README, I won't go into details here.

Design

As we all know, the RSS scenario is a typical publish-subscribe model, which is very suitable for using message queues to complete program functions. The author also considered this point first when designing.

With a message queue, we only need to focus on two major aspects - message generation and reception.

Message Generation

A simple idea is to use scheduled tasks to fetch all topic addresses from the message queue. After obtaining the feed content, check if it has been sent. If it hasn't been sent, send the content and mark it as sent.

Message Reception

After the subscriber of the message queue receives the broadcast message, the robot is responsible for sending the message.

Structure

The following is the directory structure related to the program logic:

.
├── client
│   └── qq.go # QQ robot implementation
├── config.yml # Robot configuration file
├── db.sql # Database initialization statement
├── internal
│   └── rss.go # Internal RSS update detection
├── main.go # Program entry point (initialize client, message queue, and start event monitoring)
├── message
│   └── mq.go # Message queue implementation
├── model
│   ├── conf.go # Configuration file reading
│   └── db.go # Database connection pool and a series of database operations
└── utils
    ├── error.go # Helper function for error checking
    ├── hash.go # Generate hash for a single feed content
    └── message.go # Generate messages sent by the robot

Implementation

First, implement the robot. According to our design, we can abstract the functionality of the robot as follows:

type robot interface {
	SendMessage(int64, string, bool) // Send string information to the user with the specified id (the bool value is used to indicate whether the subscriber is an individual or a group)
}

Any robot that can implement this interface can meet our requirements.

Next is the message queue. For Golang, which uses the CSP concurrency model, it is easy to implement a message queue for message transmission. The specific code can be found in message/mq.go.

Now that we have consumers and a means of transmission for messages, our ultimate goal (at least for now) is to implement the producer. The producer will retrieve the topic list (i.e., all existing URLs) from the message queue and make requests. If an update is found, it will construct a message and write the message to the corresponding topic in the message queue (the request for the topic is in internal/rss.go, and the construction of the message is in utils/message.go).

Phew~ It seems like we're done, but is that really the case? Let's examine whether the above design can meet the following requirements:

The robot responds to user commands (such as adding subscriptions, modifying subscriptions, etc.).
Every time the program starts, the robot automatically reads the previous subscription relationships.
Feed deduplication.

It can be foreseen that in order for the robot to respond to commands, it needs to listen for message events and respond accordingly. In fact, a robot often needs to start working normally after the initialization steps, so I finally designed the robot interface as follows:

type robot interface {
	Init()
	SendMessage(int64, string, bool)
	HandleEvent(*message.MessageQueue)
}

And provided an example implementation for the QQ version (in client/QQ.go).

For requirements 2 and 3, it is obvious that we need to persist the subscription relationships and the feed information that has been sent. This is where the database comes in. I stored the table creation statements in db.sql and initialized the database connection and a series of database operation methods in model/db.go.

So far, our program is basically complete.

Flaws

If you pay attention to the implementation code I mentioned above, you will easily find many problems:

Why is the receiver of the message queue not just a simple chan string, but instead wrapped in a large Receiver?
According to your logic, why do you need to start multiple goroutines for a person who subscribes to multiple URLs? Can't you just run one?
Is it really okay to couple the message queue subscription with the message reception listener?
Is there no better method for RSS deduplication than using a hash?
Will reading the database for every command affect performance?
Isn't it difficult to deploy such a simple robot with a complex database like MySQL?
Do users have to execute the database initialization statement themselves? Isn't that troublesome?
The database initialization is written in init, so it's impossible to do unit testing, right?
......

Yes, these (and even more) are all flaws in this program. In future iterations, I will do my best to fix these pain points.

Conclusion

Thank you all for enduring my rambling! Finally, please feel free to give a star to the project!~