建设电影网站数据库脚本武汉搜索引擎排名优化

1. 需求

背景：业务需要，平台将使用rocketMQ来实现消息的发送与消费，替代redis的消息功能。

需要在搭建好rocketMQ平台后，进行研究和验证。

技术：Springboot + RocketMQ5.0

使用场景：签到活动，给用户推送消息，日志上报等

2. 笔记

2.1 安装RocketMQ 5.0

2.1.1 下载

官网：https://rocketmq.apache.org/zh/docs/quickStart/01quickstart/

请注意下载二进制包，二进制包是已经编译完成后可以直接运行的，源码包是需要编译后运行的。

二进制包：https://dist.apache.org/repos/dist/release/rocketmq/5.1.0/rocketmq-all-5.1.0-bin-release.zip

2.1.2 修改参数

解压文件到 D:\rocketmq-5.0,就相当于安装好了
默认的java运行内存很大，这里要修改一下内存配置：
进入bin目录，修改runbroker.sh文件和runserver.sh（如果是windows系统，修改runbroker.cmd文件和runserver.cmd）

原本是4g,4g,2g的配置，我这里修改为了256m,256m,256m的配置，两个文件都是修改成这样就差不多了。
linux: JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"
windows: set "JAVA_OPT=%JAVA_OPT% -server -Xms256m -Xmx256m"

修改保存后，就是启动了。

2.2 启动

启动可以按照官网的quick start启动，如下：

2.2.1 本地windows启动

1. 启动NameServer

start mqnamesrv.cmd

2. 启动broker

start mqbroker -n 127.0.0.1:9876 autoCreateTopicEnable=true

3. 启动proxy

start mqbroker.cmd -n 127.0.0.1:9876

2.2.2 Linux启动

分别是在解压后的rocketMQ文件夹下执行如下命令：

1. 启动mqnamesrv（请在rocketMQ解压后的文件夹中的bin目录同级使用下面这些命令）

### 启动namesrv
$ nohup sh bin/mqnamesrv &### 验证namesrv是否启动成功
$ tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success...

2. 再启动Broker + proxy

### 先启动broker
$ nohup sh bin/mqbroker -n localhost:9876 --enable-proxy &### 验证broker是否启动成功, 比如, broker的ip是192.168.1.2 然后名字是broker-a
$ tail -f ~/logs/rocketmqlogs/broker_default.log 
The broker[broker-a,192.169.1.2:10911] boot success...

可以使用jps查看或者使用如下命令查看日志文件：

tail -f ~/logs/rocketmqlogs/broker.log

3. 如果用jps可查看启动的服务

OK，启动完成

2.3 RocketMQ Dashborad

参见git官网 https://gitcode.net/mirrors/apache/rocketmq-dashboard/

1. 本地启动rocketmq-dashboard项目

将项目拉到本地后，idea打开，修改配置文件application.yml：

rocketmq:config:# if this value is empty,use env value rocketmq.config.namesrvAddr  NAMESRV_ADDR | now, default localhost:9876# configure multiple namesrv addresses to manage multiple different clustersnamesrvAddrs:- 127.0.0.1:9876  #修改namesrv的地址- 127.0.0.2:9876

启动：

windos：启动项目,idea -> run application

linux：先用maven打成jar包，然后 java-jar 启动

2.4 rocketmq + springboot

项目结构：

1. 导入依赖

    <dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter</artifactId></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><optional>true</optional></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency><dependency><groupId>org.apache.rocketmq</groupId><artifactId>rocketmq-client-java</artifactId><version>5.0.4</version></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency></dependencies>

2. application.properties

server.port=8080#自定义proxy地址
rocketmq.proxy = 127.0.0.1:8081

3. 配置生产者

package com.example.config;import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;@Slf4j
@Configuration
public class RocketConfig {// 接入点地址，需要设置成Proxy的地址和端口列表，一般是xxx:8081;xxx:8081。@Value("${rocketmq.proxy}")private String mqProxy;@Bean(name="MyProducer")public Producer createProducer(){ClientServiceProvider provider = ClientServiceProvider.loadService();ClientConfigurationBuilder builder = ClientConfiguration.newBuilder().setEndpoints(mqProxy);ClientConfiguration configuration = builder.build();// 初始化Producer时需要设置通信配置以及预绑定的Topic。try {log.info("初始化rocketmq5.0生产者: proxy:{}",mqProxy);Producer producer = provider.newProducerBuilder().setClientConfiguration(configuration).build();log.info("初始化rocketmq5.0生产者成功: proxy:{}", mqProxy);return producer;} catch (ClientException e) {log.info("初始化rocketmq5.0生产者失败：{}", e);}return null;}
}

3. 消费者

package com.example.service;import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.consumer.ConsumeResult;
import org.apache.rocketmq.client.apis.consumer.FilterExpression;
import org.apache.rocketmq.client.apis.consumer.FilterExpressionType;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;import java.nio.charset.StandardCharsets;
import java.util.Collections;@Slf4j
@Component
public class RocketConsumer {@Value("${rocketmq.proxy}")private String mqProxy;// 为消费者指定所属的消费者分组，Group需要提前创建。private static final String My_Consumer_Group = "myConsumerGroup1";// 指定需要订阅哪个目标Topic，Topic需要提前创建。private static final String My_Topic = "myTopicTest1";@Bean(name = "MyConsumer")public void mqConsumer(){ClientServiceProvider provider = ClientServiceProvider.loadService();// 接入点地址，需要设置成Proxy的地址和端口列表，一般是xxx:8081;xxx:8081。ClientConfiguration clientConfiguration = ClientConfiguration.newBuilder().setEndpoints(mqProxy).build();// 订阅消息的过滤规则，表示订阅所有Tag的消息。String tag = "*";FilterExpression filterExpression = new FilterExpression(tag, FilterExpressionType.TAG);// 初始化PushConsumer，需要绑定消费者分组ConsumerGroup、通信参数以及订阅关系。try {log.info("构建消费者：proxy: {}, consumer_group: {}, topic: {}", mqProxy, My_Consumer_Group, My_Topic);provider.newPushConsumerBuilder().setClientConfiguration(clientConfiguration).setConsumerGroup(My_Consumer_Group)// 设置预绑定的订阅关系。.setSubscriptionExpressions(Collections.singletonMap(My_Topic, filterExpression))// 设置消费监听器。.setMessageListener(messageView -> {// 处理消息并返回消费结果。log.info("消费消息：{}", messageView);log.info("消息内容：messageId={}, messageBody={}", messageView.getMessageId(),StandardCharsets.UTF_8.decode(messageView.getBody()).toString());return ConsumeResult.SUCCESS;}).build();log.info("构建消费者成功：proxy: {}, consumer_group: {}, topic: {}", mqProxy, My_Consumer_Group, My_Topic);} catch (ClientException e) {log.info("构建消费者异常：proxy: {}, consumer_group: {}, topic: {}, Excepiton:", mqProxy, My_Consumer_Group, My_Topic, e);}}
}

5. 接口测试

package com.example.controller;import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.message.Message;
import org.apache.rocketmq.client.apis.message.MessageBuilder;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.producer.SendReceipt;
import org.apache.rocketmq.client.java.message.MessageBuilderImpl;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;import javax.annotation.Resource;
import java.nio.charset.StandardCharsets;@RestController
public class TestMqController {@Resource(name = "MyProducer")private Producer producer;@GetMapping("/sendMessage")public String sendMessage() throws ClientException {MessageBuilder messageBuilder = new MessageBuilderImpl();String msgStr = "a test message for rocketmq5.0 ...";Message message = messageBuilder.setTopic("myTopicTest1").setBody(msgStr.getBytes(StandardCharsets.UTF_8)).build();SendReceipt send = producer.send(message);return "success";}
}

6. 启动服务

7. 调接口

http://127.0.0.1:8080/sendMessage

8. 后台日志

2023-03-07 19:19:15.915  INFO 21508 --- [onsumption-1-34] com.example.service.RocketConsumer       : 消费消息：MessageViewImpl{messageId=01A87EEA967B8354040418B7B300000000, topic=myTopicTest1, bornHost=DESKTOP-RNCSLDE, bornTimestamp=1678187955816, endpoints=ipv4:127.0.0.1:8081, deliveryAttempt=1, tag=null, keys=[], messageGroup=null, deliveryTimestamp=null, properties={}}
2023-03-07 19:19:15.915  INFO 21508 --- [onsumption-1-34] com.example.service.RocketConsumer       : 消息内容：messageId=01A87EEA967B8354040418B7B300000000, messageBody=a test message for rocketmq5.0 ...

9. dashborad

本地启动rocketmq dashborad, 修改服务启动端口 server.port: 8088

访问面板：http://127.0.0.1:8088/#/ （我调用了4次接口）

3. 思考

1. 消费异常如何处理？
   • 打印日志，记录msgId, body等
   • 重试机制是怎样的
2. 之前的消息是存在redis，消息如何从redis平滑迁移到rocketmq

代码参考文章：RocketMQ 5.0 实战