FastDFS 服务器

2021-05-14 约 4044 字 阅读时长9 分钟
目录

FastDFS

FastDFS入门

FastDFS 是一个开源的轻量级分布式文件系统

FastDFS服务端由两个部分组成：跟踪器（tracker）、存储节点（storage）

  • 跟踪器：主要是调度工作，在内存中记录存储节点(storage)的状态信息，是前端 Client 和后端存储节点 storage 的枢纽。Tracker server的性能非常高，一个大的集群（上百个group）中有3台就足够了
  • 存储节点：用于存储文件，包括文件和文件属性(meta data) 都保存到存储服务器磁盘上，完成文件管理的所有功能：文件存储、文件同步和提供文件访问

安装

  1. 检查机器是否包含如下依赖：gcc、libevent、libevent-devel

    bash 
    1#安装 gcc、libevent、libevent-devel
2yum install gcc libevent libevent-devel -y

  2. github地址：https://github.com/happyfish100，下载安装：libfastcommon库、FastDFS-master

    bash 
    1#解压libfastcommon库、FastDFS-master
2unzip libfastcommon-master.zip
3unzip FastDFS-master.zip
4
5#进入相应的目录编译安装
6./make.sh
7./make.sh install

  3. 查看安装文件

    bash 
    1#进入 /usr/bin，查看fdfs相关命令
2ll | grep fdfs
3
4#进入 /etc/fdfs，查看fdfs相关配置文件
5ll
6-rw-r--r-- 1 root root  1909 Aug 15 19:52 client.conf   #客户端配置
7-rw-r--r-- 1 root root 10246 Aug 15 19:52 storage.conf   #存储节点配置
8-rw-r--r-- 1 root root   620 Aug 15 19:52 storage_ids.conf   #存储节点id
9-rw-r--r-- 1 root root  9138 Aug 15 19:52 tracker.conf   #跟踪器配置

  4. 拷贝FastDFS-master/conf目录下的http.conf、 mime.types到配置目录 /etc/fdfs

    bash 
    1cp http.conf /etc/fdfs/
2cp mime.types /etc/fdfs/

启动

  1. 修改必要配置：日志、文件存放位置，并手动创建相应的存放目录

    bash 
     1#备份配置文件
 2cp tracker.conf tracker.conf.bak
 3cp storage.conf storage.conf.bak
 4
 5#修改tracker.conf配置文件中，store和log存放位置
 6# the base path to store data and log files
 7base_path = /opt/FastDFS/tracker
 8
 9#修改storage.conf配置文件中，sotre和log存放位置、file存放位置、tracerIp
10# the base path to store data and log files
11base_path = /opt/FastDFS/storeage
12# store_path#, based on 0, to configure the store paths to store files
13store_path0 = /opt/FastDFS/storeage/files
14# tracker_server can ocur more than once for multi tracker servers.
15tracker_server = 127.0.0.1:22122

  2. 使用命令启动跟踪器、存储节点

    bash 
     1#启动 跟踪器
 2fdfs_trackerd /etc/fdfs/tracker.conf start
 3
 4#启动 存储节点
 5fdfs_storaged /etc/fdfs/storage.conf start
 6
 7#查看启动
 8ps -ef | grep fdfs
 9root        10     1  0 21:19 ?        00:00:00 fdfs_trackerd tracker.conf start
10
11#发现存储节点启动没成功，查看logs
12$cat opt/fdfs/storage/logs/storaged.log
13[2021-08-15 21:26:04] ERROR - file: fdfs_shared_func.c, line: 474, host "127.0.0.1" is invalid, error in              fo: Unknown host
14[2021-08-15 21:26:04] CRIT - exit abnormally!
15
16#发现tracerIp配置错误，修改后重新启动，并查看启动情况
17$ps -ef | grep fdfs
18root        10     1  0 21:19 ?        00:00:00 fdfs_trackerd /etc/fdfs/tracker.conf start
19root      5122     1 30 21:32 ?        00:00:13 fdfs_storaged /etc/fdfs/storage.conf start

  3. 服务关闭

    bash 
    1#关闭跟踪器
2fdfs_trackerd /etc/fdfs/tracker.conf stop
3
4#关闭存储节点
5fdfs_storaged /etc/fdfs/storage.conf stop

FastDFS测试

  1. 测试时需要配置client.conf文件中的 traverIp和base_path

    ini 
    1# the base path to store log files
2base_path = /opt/FastDFS/client
3
4# tracker_server can ocur more than once for multi tracker servers.
5tracker_server = 192.168.45.12:22122

  2. 测试上传

    bash 
     1#上传
 2fdfs_test /etc/fdfs/client.conf upload 1.txt
 3#group_name 组名   remote_filename M00/:配置的path，00/00/：磁盘路径，wKgtDGE...：文件名
 4group_name=group1, remote_filename=M00/00/00/wKgtDGEZI7uAMqbkAAAACpEbkw40686912
 5
 6#查看上传文件
 7ll /opt/FastDFS/storeage/files/data/00/00/
 8wKgtDGEZI7uAMqbkAAAACpEbkw406869121.txt		#上传的文件
 9wKgtDGEZI7uAMqbkAAAACpEbkw406869121.txt-m	#上传的文件属性
10wKgtDGEZI7uAMqbkAAAACpEbkw40686912_big1.txt	#上传的文件备份
11wKgtDGEZI7uAMqbkAAAACpEbkw40686912_big1.txt-m

  3. 测试下载

    bash 
    1fdfs_test /etc/fdfs/client.conf download group1 M00/00/00/wKgtDGEZI7uAMqbkAAAACpEbkw40686912.txt

  4. 测试删除

    bash 
    1fdfs_test /etc/fdfs/client.conf delete group1 M00/00/00/wKgtDGEZI7uAMqbkAAAACpEbkw40686912.txt_big

分布式文件系统http访问

nginx的FastDFS扩展模块整合

  1. FastDFS github下载FastDFS-nginx-module-master，并解压

    bash 
    1unzip FastDFS-nginx-module-master.zip

  2. 安装nginx，添加 FastDFS 模块

    bash 
     1tar -zxvf nginx-1.20.1.tar.gz
 2
 3cd nginx-1.20.1/
 4
 5#debain系列下可能需要安装先运行如下命令安装依赖
 6apt-get install libpcre3 libpcre3-dev 
 7
 8./configure --prefix=/etc/nginx-fdfs --sbin-path=/usr/bin/nginx-fdfs --add-module=/root/FastDFS-nginx-module-master/src/
 9
10make
11
12make install

  3. 配置模块

    ini 
     1#将模块配置文件拷贝到 fdfs 配置目录下
 2cp FastDFS-nginx-module-master/src/mod_FastDFS.conf /etc/fdfs/
 3
 4#修改配置文件，基础目录（该目录必须已存在）、url带组名访问、跟踪器
 5# the base path to store log files
 6base_path=/opt/FastDFS/FastDFS_mod
 7# if the url / uri including the group name
 8url_have_group_name = true
 9# FastDFS tracker_server can ocur more than once, and tracker_server
10tracker_server=192.168.45.12:22122
11# store_path must same as storage.conf
12store_path0=/opt/FastDFS/storeage/files

  4. 配置nginx

    ini 
    1#拦截请求文件的路径，并使用FastDFS的nginx模块进行转发
2location ~/group[1-9]/M0[0-9]{
3	ngx_FastDFS_module;
4}

  5. 启动nginx

    bash 
    1#检测配置文件是否正确
2nginx-fdfs -c /etc/nginx-fdfs/conf/nginx.conf -t
3
4#指定配置文件启动
5nginx-fdfs -c /etc/nginx-fdfs/conf/nginx.conf
6
7#查看启动状态
8ps -ef | grep nginx

  6. 浏览器访问：http://127.0.0.1/group1/M00/00/00/wKgtDGEZI7uAMqbkAAAACpEbkw40686912

扩展模块执行流程

image-20210815225946104

FastDFS在java中使用

  1. 下载happyfish100/FastDFS-client-java java 操作 FastDFS 源代码

  2. FastDFS-client-java是一个普通的maven项目，使用 maven install 安装到本地仓库

  3. 创建普通的 maven 项目，添加依赖

    xml 
    1<!-- 该依赖为刚刚maven安装的 -->
2<dependency>
3    <groupId>org.csource</groupId>
4    <artifactId>FastDFS-client-java</artifactId>
5    <version>1.29-SNAPSHOT</version>
6</dependency>

  4. 创建配置文件，FastDFS.properties

    properties 
    1FastDFS.tracker_servers = 192.168.45.185:22122,192.168.10.130:22122
2FastDFS.connection_pool.enabled = true
3FastDFS.connection_pool.max_count_per_entry = 500
4FastDFS.connection_pool.max_idle_time = 3600
5FastDFS.connection_pool.max_wait_time_in_ms = 1000

  5. 创建 FastDFSUtil 工具类

    java 
     1public class FastDFSUtil {
 2    private static TrackerServer trackerServer=null;  //tracker 服务器对象
 3    private static StorageServer storageServer=null;  //storage 服务器对象
 4    private static TrackerClient trackerClient=null;  //tracker 客户端对象
 5    private static StorageClient storageClient=null;  //storage 客户端对象
 6    private FastDFSUtil(){};
 7    static {
 8        try {
 9            // 加载Properties配置文件
10            ClientGlobal.initByProperties("FastDFS.properties"); 
11            trackerClient=new TrackerClient();
12            trackerServer=trackerClient.getTrackerServer();
13            storageServer=trackerClient.getStoreStorage(trackerServer);
14            storageClient=new StorageClient(trackerServer,storageServer);
15        } catch (IOException e) {
16            e.printStackTrace();
17        } catch (MyException e) {
18            e.printStackTrace();
19        }
20    }
21
22    /**
23     * 适合上传本地图片，web上传，可以使用 uploadFile 重载方法，上传byte数组
24     * @param localFilename 本地文件名
25     * @param extName 文件扩展名
26     * @param metas 文件属性
27     * @return {@link String[]}  返回文件存储在 FastDFS 中信息，很重要
28     */
29    public static String[] uploadFile(String localFilename, String extName, NameValuePair[] metas) throws Exception {
30        return storageClient.upload_file(localFilename,extName,metas);
31    }
32
33    /**
34     * 直接下载文件到本地，web程序可以使用download_file的重载方法，可以返回byte数组
35     * @param groupName  在fdfs服务器中存储的组，文件上传时会返回
36     * @param remoteName 在fdfs服务器中存储的具体位置，文件上传时会返回
37     * @param localName  下载到本地存储的名称
38     * @return int 返回0，代表下载成功
39     */
40    public static int downloadFile(String groupName,String remoteName,String localName) throws Exception {
41        return storageClient.download_file(groupName,remoteName,localName);
42    }
43
44    /**
45     * 文件删除
46     * @param groupName 在fdfs服务器中存储的组，文件上传时会返回
47     * @param remoteName 在fdfs服务器中存储的具体位置，文件上传时会返回
48     * @return int 返回0代表删除成功
49     * @throws Exception
50     */
51    public static int delFile(String groupName,String remoteName) throws Exception {
52        return storageClient.delete_file(groupName,remoteName);
53    }
54}

  6. 创建测试Demo

    java 
     1public static void main(String[] args) throws Exception{
 2        //上传
 3    String[] result=FastDFSUtil.uploadFile("E:\\123.jpg","jpg",null);
 4    Arrays.stream(result).forEach(System.out::println);
 5    // result值为：["group1","M00/00/00/wKgtuWEhM7WAG15kAAYIjBrlJRI160.jpg"]
 6
 7    String group="group1";
 8    String remote="M00/00/00/wKgtuWEhM7WAG15kAAYIjBrlJRI160.jpg";
 9
10    //下载
11    FastDFSUtil.downloadFile(group, remote, "e:\\1.jpg");
12
13    //删除
14    FastDFSUtil.delFile(group,remote);
15}

注意：

需要将文件上传后返回的 String 数组保存好（一般存在数据库），它包含了上传文件在 FastDFS 服务器中存储的相关信息

FastDFS在web中使用

  1. 添加依赖，fileupload和FastDFS

    xml 
     1<dependency>
 2    <groupId>commons-fileupload</groupId>
 3    <artifactId>commons-fileupload</artifactId>
 4    <version>1.3.3</version>
 5</dependency>
 6<dependency>
 7    <groupId>org.csource</groupId>
 8    <artifactId>FastDFS-client-java</artifactId>
 9    <version>1.29-SNAPSHOT</version>
10</dependency>

  2. 注册 StorageClient bean

    java 
    1@Bean
2public StorageClient storageClient() throws MyException, IOException {
3    ClientGlobal.initByProperties("FastDFS.properties");
4    TrackerClient trackerClient=new TrackerClient();
5    TrackerServer trackerServer=trackerClient.getTrackerServer();
6    StorageServer storageServer=trackerClient.getStoreStorage(trackerServer);
7    StorageClient storageClient=new StorageClient(trackerServer,storageServer);
8    return storageClient;
9}

  3. 编写文件上传控制器

    java 
     1@RestController
 2@RequestMapping("/file")
 3public class FileController {
 4
 5    @Autowired
 6    private StorageClient client;
 7
 8    @PostMapping("/upload")
 9    public Object upload(@RequestParam("files") MultipartFile[] files, ) throws Exception {
10        if (files==null||files.length==0){
11            return ResponWrapper.setSuccess(false);
12        }
13        MultipartFile file=files[0];
14        String fileName=file.getOriginalFilename();
15        int extIndex=fileName.lastIndexOf(".")+1;
16        String fileExtName="";
17        if (extIndex!=0){
18            fileExtName=fileName.substring(extIndex);
19        }
20        String[] result=client.upload_file(file.getBytes(),fileExtName,null);
21        if (result.length!=2){
22            return ResponWrapper.setSuccess(false);
23        }
24        FileDto fileDto=new FileDto();
25        fileDto.setGroupName(result[0]);
26        fileDto.setRemoteName(result[1]);
27        return ResponWrapper.setSuccess(true);
28    }
29
30    @RequestMapping("/download")
31    public void download(String id, HttpServletResponse response) throws Exception {
32        FileDto fileDto=fileService.selectById(id);
33        byte[] bytes=client.download_file(fileDto.getGroupName(),fileDto.getRemoteName());
34        response.reset(); //设置页面不缓存，清空buffer
35        response.setCharacterEncoding("UTF-8");  //设置字符编码
36        response.setContentType("multipart/form-data"); //设置二进制传输数据
37        //  设置响应头
38        response.setHeader("Content-Disposition", "attachment;fileName=" + fileDto.getId());
39        response.getOutputStream().write(bytes);
40        response.getOutputStream().flush();
41    }
42    @RequestMapping("/delete")
43    public ResponWrapper delete(String fid,String uid) throws Exception {
44        FileDto fileDto=fileService.selectById(fid);
45        int res=client.delete_file(fileDto.getGroupName(),fileDto.getRemoteName());
46        fileService.deleteById(uid,fid);
47        return ResponWrapper.setSuccess(true);
48    }
49}

Spring文件上传大小配置

properties 
1#springMVC允许上传单个文件最大大小
2spring.servlet.multipart.max-file-size=1MB
3#spring允许表单上传最大大小
4spring.servlet.multipart.max-request-size=10MB

FastDFS集群部署

image-20211006035144414

部署FastDFS集群，架构图如上，需准备7台服务器

其中 group1 有两台主机，group2 有两台主机；两台 tracker主机，需要安装 nginx，方便web访问；一台nginx主机，用作统一入口，并实现负载均衡

  1. 安装linux虚拟机（7台）

  2. 安装必要软件

    bash 
    1#lrzsz 为linux上传下载文件使用的工具
2#rz 上传
3#sz 下载
4yum install perl gcc libevent libevent-devel unzip lrzsz -y

  3. 安装 tracker

    bash 
     1#安装fdfs公共库
 2unzip libfastcommon-master.zip && cd ibfastcommon-master
 3./make.sh
 4./make.sh install
 5
 6#安装fdfs
 7unzip FastDFS-master.zip && cd FastDFS-master
 8./make.sh
 9./make.sh install
10
11#拷贝必要配置文件
12cp FastDFS-master/conf/http.conf /etc/fdfs/
13cp FastDFS-master/conf/mime.types /etc/fdfs/
14
15#修改tracker配置文件
16base_path = /opt/FastDFS/tracker
17
18#启动tracker
19fdfs_trackerd /etc/fdfs/tracker.conf start
20
21#检查是否成功启动
22ps -ef | grep fdfs

  4. 安装storage

    bash 
     1#安装fdfs公共库
 2unzip libfastcommon-master.zip && cd ibfastcommon-master
 3./make.sh
 4./make.sh install
 5
 6#安装fdfs
 7unzip FastDFS-master.zip && cd FastDFS-master
 8./make.sh
 9./make.sh install
10
11#拷贝必要配置文件
12cp FastDFS-master/conf/http.conf /etc/fdfs/
13cp FastDFS-master/conf/mime.types /etc/fdfs/
14
15#修改storage配置文件
16group_name = group1 / group_name = group2
17base_path = /opt/FastDFS/storeage
18store_path0 = /opt/FastDFS/storeage/files
19tracker_server = 192.168.10.129:22122
20tracker_server = 192.168.10.130:22122
21
22#启动storage
23fdfs_storaged /etc/fdfs/storage.conf start
24
25#检查是否成功启动
26ps -ef | grep fdfs

  5. 关闭主机防火墙，修改tracker负载均衡策略，并使用java客户端测试程序

    bash 
     1#关闭防火墙
 2systemctl status firewall	#查看防火墙状态
 3systemctl stop firewalld	#关闭防火墙
 4systemctl start firewalld	#启动防火墙
 5systemctl stop firewalld	#重启防火墙
 6
 7#修改 tracker 配置文件负载策略
 8# 0: round robin
 9# 1: specify group
10# 2: load balance, select the max free space group to upload file
11store_lookup = 0
12
13#java客户端测试，tracker负载策略改为0时，此时tracker应轮流上传到两个group

  6. 配置trcker 主机上的 nginx

    bash 
     1#单纯的为了负载均衡，配置如下
 2#安装nginx
 3tar -zxvf nginx-1.20.1.tar.gz & cd nginx-1.20.1/
 4./configure --prefix=/usr/local/nginx
 5make
 6make install
 7
 8#修改nginx配置文件
 9upstream myweb {
10    server 192.168.10.131:80;
11    server 192.168.10.132:80;
12    server 192.168.10.133:80;
13    server 192.168.10.134:80;
14}
15location ~/group[1-9]/M0[0-9]{
16	proxy_pass http://myweb;
17}
18
19#启动，加-t参数，校验配置文件
20/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
21
22#检测是否启动成功
23ps -ef | grep nginx

  7. 配置storage nginx

    bash 
     1#安装fdfs_nginx
 2unzip FastDFS-nginx-module-master.zip
 3tar -zxvf nginx-1.20.1.tar.gz & cd nginx-1.20.1/
 4./configure --prefix=/etc/nginx-fdfs --sbin-path=/usr/bin/nginx-fdfs --add-module=/root/FastDFS-nginx-module-master/src/
 5make
 6make install
 7cp FastDFS-nginx-module-master/src/mod_FastDFS.conf /etc/fdfs/
 8
 9#配置mod_FastDFS.conf配置文件,group_name针对不同的当前storage所属的group配置
10base_path=/opt/FastDFS/FastDFS_mod
11group_name=group1	/ group_name=group2
12store_path0=/opt/FastDFS/storeage/files
13url_have_group_name = true
14tracker_server=192.168.10.130:22122
15tracker_server=192.168.10.129:22122
16group_count = 2
17[group1]
18group_name=group1
19storage_server_port=23000
20store_path_count=1
21store_path0=/opt/FastDFS/storeage/files
22[group2]
23group_name=group2
24storage_server_port=23000
25store_path_count=1
26store_path0=/opt/FastDFS/storeage/files
27
28#配置nginx配置文件
29location ~/group[1-9]/M0[0-9]{
30	ngx_FastDFS_module;
31}
32
33#启动，加-t参数，校验配置文件
34nginx-fdfs -c /etc/nginx-fdfs/conf/nginx.conf
35
36#检测是否启动成功
37ps -ef | grep nginx

  8. 配置nginx负载均衡，统一入口

    bash 
     1#安装nginx
 2#配置nginx配置文件
 3upstream myweb {
 4    server 192.168.10.129:80;
 5    server 192.168.10.130:80;
 6}
 7location ~/group[1-9]/M0[0-9]{
 8	proxy_pass http://myweb;
 9}
10
11#启动nginx，并在浏览器访问
12/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
13
14#浏览器访问：http://192.168.10.128/group1/M00/00/00/wKgKhWFcpDSAH-LoAAPNBgx7Frs765.jpg
15#可以看到文件

  9. 分析运行流程

    mermaid 
    1graph TD
2A[nginx] -->|1.携带地址负载访问带扩展模块的nginx|B[nginx_fdfs & storage]
3    B --> |2.根据扩展模块携带地址访问Tracker|C[tracker]
4    C --> |3.根据地址返回对应的Storage的ip|B
5    B --> |4.根据tracker返回的IP地址找到对应的nginx_fdfs|B 
6    B --> |5.返回文件流|A
使用滚轮缩放
按住拖动