FastDFS安装配置,fastDFS分布式文件系统(一)

位置:首页>文章>详情   分类: 教程分享 > Java教程   阅读(1491)   2023-03-28 11:29:14

什么是FastDFS?

  FastDFS的开发者是这样介绍的:
  FastDFS is an open source high performance distributed file system (DFS). It's major functions include: file storing, file syncing and file accessing, and design for high capacity and load balance.
  FastDFS是一个开源高性能分布式文件系统(DFS)。它的主要功能包括:文件存储、文件同步和文件访问,以及高容量和负载平衡的设计。


一、获取需要的安装文件

1.1获取安装FastDFS依赖的公共函数库

下载地址: https://github.com/happyfish100/libfastcommon/archive/V1.0.35.tar.gz

系统中执行命令:

[root@localhost fastdfs-libs]# wget https://github.com/happyfish100/libfastcommon/archive/V1.0.35.tar.gz

--2017-07-20 09:03:34--  https://github.com/happyfish100/libfastcommon/archive/V1.0.35.tar.gz

Resolving github.com... 192.30.255.113, 192.30.255.112

Connecting to github.com|192.30.255.113|:443... connected.

HTTP request sent, awaiting response... 302 Found

Location: https://codeload.github.com/happyfish100/libfastcommon/tar.gz/V1.0.35 [following]

--2017-07-20 09:03:36--  https://codeload.github.com/happyfish100/libfastcommon/tar.gz/V1.0.35

Resolving codeload.github.com... 192.30.255.121, 192.30.255.120

Connecting to codeload.github.com|192.30.255.121|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: 434734 (425K) [application/x-gzip]

Saving to: “V1.0.35”

 

100%[====================================================================================================================================================================================================================================>] 434,734     47.0K/s   in 9.7s   

2017-07-20 09:03:47 (44.0 KB/s) - “V1.0.35” saved [434734/434734]

[root@localhost fastdfs-libs]#

1.2获取FastDFS安装文件

下载地址: https://github.com/happyfish100/fastdfs/archive/V5.10.tar.gz

下载命令:

[root@localhost fastdfs-libs]# wget https://github.com/happyfish100/fastdfs/archive/V5.10.tar.gz

--2017-07-20 09:06:41--  https://github.com/happyfish100/fastdfs/archive/V5.10.tar.gz

Resolving github.com... 192.30.255.112, 192.30.255.113

Connecting to github.com|192.30.255.112|:443... connected.

HTTP request sent, awaiting response... 302 Found

Location: https://codeload.github.com/happyfish100/fastdfs/tar.gz/V5.10 [following]

--2017-07-20 09:06:43--  https://codeload.github.com/happyfish100/fastdfs/tar.gz/V5.10

Resolving codeload.github.com... 192.30.255.120, 192.30.255.121

Connecting to codeload.github.com|192.30.255.120|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: unspecified [application/x-gzip]

Saving to: “V5.10”

 

    [                                        <=>                                                                                                                                                                                          ] 336,589     15.5K/s   in 15s    

 

2017-07-20 09:06:59 (22.5 KB/s) - “V5.10” saved [336589]

[root@localhost fastdfs-libs]#

1.3解压安装包

1.3.1查看刚才下载的两个文件

[root@localhost fastdfs-libs]# ll

total 760

-rw-r--r--. 1 root root 434734 Jul 20 09:03 V1.0.35

-rw-r--r--. 1 root root 336589 Jul 20 09:06 V5.10

[root@localhost fastdfs-libs]#

说明:由于wget工具去掉了文件格式,所以文件名就成上面这个样子了。其中

V1.0.35libfastcommon公共函数库安装包

V5.10FastDFS安装包

解压命令:

[root@localhost fastdfs-libs]# tar -xzvf V5.10

[root@localhost fastdfs-libs]# tar -xzvf V1.0.35

解压后目录中的文件:

[root@localhost fastdfs-libs]# ll

total 768

drwxrwxr-x. 10 root root   4096 Apr  5 18:38 fastdfs-5.10

drwxrwxr-x.  5 root root   4096 Jul 20 09:23 libfastcommon-1.0.35

-rw-r--r--.  1 root root 434734 Jul 20 09:03 V1.0.35

-rw-r--r--.  1 root root 336589 Jul 20 09:06 V5.10

[root@localhost fastdfs-libs]#

二、安装libfastcommon函数库

注意:FastDFS版本5.05以前不需要安装,如果版本低于5.05则忽略该步骤

切换至libfastcommon目录内

命令:

[root@localhost fastdfs-libs]# ll

total 764

drwxrwxr-x. 5 root root   4096 Mar  2 18:47 libfastcommon-1.0.35

-rw-r--r--. 1 root root 434734 Jul 20 09:03 V1.0.35

-rw-r--r--. 1 root root 336589 Jul 20 09:06 V5.10

[root@localhost fastdfs-libs]# cd libfastcommon-1.0.35/

[root@localhost libfastcommon-1.0.35]#

编译文件

命令

[root@localhost libfastcommon-1.0.35]# ./make.sh

o fast_task_queue.lo fast_timer.lo process_ctrl.lo fast_mblock.lo connection_pool.lo fast_mpool.lo fast_allocator.lo fast_buffer.lo multi_skiplist.lo flat_skiplist.lo system_info.lo fast_blocked_queue.lo id_generator.lo char_converter.lo char_convert_loader.lo -lm -lpthread

ar rcs libfastcommon.a hash.o chain.o shared_func.o ini_file_reader.o logger.o sockopt.o base64.o sched_thread.o http_func.o md5.o pthread_func.o local_ip_func.o avl_tree.o ioevent.o ioevent_loop.o fast_task_queue.o fast_timer.o process_ctrl.o fast_mblock.o connection_pool.o fast_mpool.o fast_allocator.o fast_buffer.o multi_skiplist.o flat_skiplist.o system_info.o fast_blocked_queue.o id_generator.o char_converter.o char_convert_loader.o

[root@localhost libfastcommon-1.0.35]#

注意,这里的并不是通用的系统make命令,而是执行目录里面的make.sh脚本

安装文件

命令:

[root@localhost libfastcommon-1.0.35]# ./make.sh install

mkdir -p /usr/lib64

mkdir -p /usr/lib

install -m 755 libfastcommon.so /usr/lib64

install -m 755 libfastcommon.so /usr/lib

mkdir -p /usr/include/fastcommon

install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_define.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h fast_mpool.h fast_allocator.h fast_buffer.h skiplist.h multi_skiplist.h flat_skiplist.h skiplist_common.h system_info.h fast_blocked_queue.h php7_ext_wrapper.h id_generator.h char_converter.h char_convert_loader.h /usr/include/fastcommon

[root@localhost libfastcommon-1.0.35]#

如果安装提示缺少gcc编译器,使用yum安装,

命令:

yum -y install gcc-c++

三、安装FastDFS

切换至FastDFS安装文件目录

命令:

[root@localhost fastdfs-libs]# cd fastdfs-5.10/

[root@localhost fastdfs-5.10]#

3.1编译FastDFS

命令:

[root@localhost fastdfs-5.10]# ./make.sh

3.2安装FastDFS

命令:

[root@localhost fastdfs-5.10]# ./make.sh install

mkdir -p /usr/bin

mkdir -p /etc/fdfs

cp -f fdfs_trackerd /usr/bin

if [ ! -f /etc/fdfs/tracker.conf.sample ]; then cp -f ../conf/tracker.conf /etc/fdfs/tracker.conf.sample; fi

if [ ! -f /etc/fdfs/storage_ids.conf.sample ]; then cp -f ../conf/storage_ids.conf /etc/fdfs/storage_ids.conf.sample; fi

mkdir -p /usr/bin

mkdir -p /etc/fdfs

cp -f fdfs_storaged  /usr/bin

if [ ! -f /etc/fdfs/storage.conf.sample ]; then cp -f ../conf/storage.conf /etc/fdfs/storage.conf.sample; fi

mkdir -p /usr/bin

mkdir -p /etc/fdfs

mkdir -p /usr/lib64

mkdir -p /usr/lib

cp -f fdfs_monitor fdfs_test fdfs_test1 fdfs_crc32 fdfs_upload_file fdfs_download_file fdfs_delete_file fdfs_file_info fdfs_appender_test fdfs_appender_test1 fdfs_append_file fdfs_upload_appender /usr/bin

if [ 0 -eq 1 ]; then cp -f libfdfsclient.a /usr/lib64; cp -f libfdfsclient.a /usr/lib/;fi

if [ 1 -eq 1 ]; then cp -f libfdfsclient.so /usr/lib64; cp -f libfdfsclient.so /usr/lib/;fi

mkdir -p /usr/include/fastdfs

cp -f ../common/fdfs_define.h ../common/fdfs_global.h ../common/mime_file_parser.h ../common/fdfs_http_shared.h ../tracker/tracker_types.h ../tracker/tracker_proto.h ../tracker/fdfs_shared_func.h ../storage/trunk_mgr/trunk_shared.h tracker_client.h storage_client.h storage_client1.h client_func.h client_global.h fdfs_client.h /usr/include/fastdfs

if [ ! -f /etc/fdfs/client.conf.sample ]; then cp -f ../conf/client.conf /etc/fdfs/client.conf.sample; fi

1

上方没有报错则安装成功。

 

3.3sample

切换至目录/etc/fast/查看

命令:

[root@localhost fastdfs-5.10]# cd /etc/fdfs

[root@localhost fdfs]# ll

total 24

-rw-r--r--. 1 root root 1461 Jul 20 09:29 client.conf.sample

-rw-r--r--. 1 root root 7927 Jul 20 09:29 storage.conf.sample

-rw-r--r--. 1 root root  105 Jul 20 09:29 storage_ids.conf.sample

-rw-r--r--. 1 root root 7389 Jul 20 09:29 tracker.conf.sample

[root@localhost fdfs]#

 

如上,安装成功后就会生成如上的4.sample文件(示例配置文件),我们再分别拷贝出4个后面用的正式的配置文件:

[root@localhost fdfs]# cp client.conf.sample client.conf

[root@localhost fdfs]# cp storage.conf.sample storage.conf

[root@localhost fdfs]# cp tracker.conf.sample tracker.conf

[root@localhost fdfs]# ll

total 44

-rw-r--r--. 1 root root 1461 Jul 20 09:37 client.conf

-rw-r--r--. 1 root root 1461 Jul 20 09:29 client.conf.sample

-rw-r--r--. 1 root root 7927 Jul 20 09:37 storage.conf

-rw-r--r--. 1 root root 7927 Jul 20 09:29 storage.conf.sample

-rw-r--r--. 1 root root  105 Jul 20 09:29 storage_ids.conf.sample

-rw-r--r--. 1 root root 7389 Jul 20 09:37 tracker.conf

-rw-r--r--. 1 root root 7389 Jul 20 09:29 tracker.conf.sample

[root@localhost fdfs]#

 

至此FastDFS已经安装完毕,接下来就是相关配置

 

四、FastDFS配置

4.1配置Tracker

在配置Tracker之前,首先需要创建Tracker服务器的文件路径,即用于存储Tracker的数据文件和日志文件等,我这里选择在/opt目录下创建一个fastdfs_tracker目录用于存放Tracker服务器的相关文件:

命令:

[root@localhost opt]# mkdir -p  /opt/fastdfs_tracker

接下来就要重新编辑上一步准备好的/etc/fdfs目录下的tracker.conf配置文件,打开文件后依次做以下修改:

  1. disabled=false #启用配置文件(默认启用)
  2. port=22122 #设置tracker的端口号,通常采用22122这个默认端口
  3. base_path=/opt/fastdfs_tracker #设置tracker的数据文件和日志目录
  4. http.server_port=8800 #设置http端口号,默认为8080

2

配置完成后就可以启动Tracker服务器了,但首先依然要为启动脚本创建软引用,因为fdfs_trackerd等命令在/usr/local/bin中并没有,而是在/usr/bin路径下:

[root@localhost fdfs]# ln -s /usr/bin/fdfs_trackerd /usr/local/bin

[root@localhost fdfs]# ln -s /usr/bin/stop.sh /usr/local/bin

[root@localhost fdfs]# ln -s /usr/bin/restart.sh /usr/local/bin

[root@localhost fdfs]#

3

最后通过命令启动Tracker服务器:

[root@localhost fdfs]# service fdfs_trackerd start

Starting FastDFS tracker server:

[root@localhost fdfs]# ps -ef|grep tracker

root       8649      1  0 09:44 ?        00:00:00 /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf

root       8657   2652  0 09:44 pts/0    00:00:00 grep tracker

[root@localhost fdfs]#

如果启动命令执行成功,那么同时在刚才创建的tracker文件目录/opt/fastdfs_tracker中就可以看到启动后新生成的datalogs目录,tracker服务的端口也应当被正常监听,最后再通过netstat命令查看一下端口监听情况:

4

[root@localhost fdfs]# netstat -unltp|grep fdfs

tcp        0      0 0.0.0.0:22122               0.0.0.0:*                   LISTEN      8649/fdfs_trackerd 

[root@localhost fdfs]#

可以看到tracker服务运行的22122端口正常被监听

确认tracker正常启动后可以将tracker设置为开机启动,打开/etc/rc.d/rc.local并在其中加入以下配置

service fdfs_trackerd start

如果重启后发现未能自动启动则通过命令ll /etc/rc.d/rc.local检查一下rc.local是否具备可执行权限,若是无可执行权限则通过chmod +x /etc/rc.d/rc.local进行授权

Tracker至此就配置好了

4.2配置Storage

同理,步骤基本与配置Tracker一致,首先是创建Storage服务器的文件目录,需要注意的是同Tracker相比我多建了一个目录,因为Storage还需要一个文件存储路径,用于存放接收的文件:

创建目录命令:

[root@localhost fdfs]# mkdir /opt/fastdfs_storage

[root@localhost fdfs]# mkdir /opt/fastdfs_storage_data

接下来修改/etc/fdfs目录下的storage.conf配置文件,打开文件后依次做以下修改:

  1. disabled=false #启用配置文件(默认启用)
  2. group_name=group1 #组名,根据实际情况修改
  3. port=23000 #设置storage的端口号,默认是23000,同一个组的storage端口号必须一致
  4. base_path=/opt/fastdfs_storage #设置storage数据文件和日志目录
  5. store_path_count=1 #存储路径个数,需要和store_path个数匹配
  6. store_path0=/opt/fastdfs_storage_data #实际文件存储路径
  7. tracker_server= 192.168.8.202:22122 #tracker 服务器的 IP地址和端口号,如果是单机搭建,IP不要写127.0.0.1,否则启动不成功(此处的ip是我的CentOS虚拟机ip
  8. http.server_port=8888 #设置 http 端口号

上述红色代表修改部分

配置完成后同样要为Storage服务器的启动脚本设置软引用:

[root@localhost fdfs]# ln -s /usr/bin/fdfs_storaged /usr/local/bin

[root@localhost fdfs]#

接下来就可以启动Storage服务了:

[root@localhost fdfs]# service fdfs_storaged start

Starting FastDFS storage server:

[root@localhost fdfs]# ps -ef |grep storage

root       8677      1 26 09:58 ?        00:00:03 /usr/bin/fdfs_storaged /etc/fdfs/storage.conf

root       8687   2652  0 09:58 pts/0    00:00:00 grep storage

[root@localhost fdfs]#

同理,如果启动成功,/opt/fastdfs_storage中就可以看到启动后新生成的datalogs目录,端口23000也应被正常监听,还有一点就是文件存储路径下会生成多级存储目录,那么接下来看看是否启动成功了:

5

端口监听情况:

[root@localhost data]# netstat -unltp|grep fdfs

tcp        0      0 0.0.0.0:23000               0.0.0.0:*                   LISTEN      8677/fdfs_storaged 

tcp        0      0 0.0.0.0:22122               0.0.0.0:*                   LISTEN      8649/fdfs_trackerd 

[root@localhost data]#

6

如上图,可以看到此时已经正常监听tracker22122端口和storage23000端口,至此storage服务器就已经配置完成,确定了storage服务器启动成功后,还有一项工作就是看看storage服务器是否已经登记到 tracker服务器(也可以理解为trackerstorage是否整合成功),运行以下命令:

[root@localhost data]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf

[2017-07-20 10:03:17] DEBUG - base_path=/opt/fastdfs_storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

 

server_count=1, server_index=0

 

tracker server is 192.168.8.202:22122

 

group count: 1

 

Group 1:

group name = group1

disk total space = 26917 MB

disk free space = 21646 MB

trunk free space = 0 MB

storage server count = 1

active server count = 1

storage server port = 23000

storage HTTP port = 8888

store path count = 1

subdir count per path = 256

current write server index = 0

current trunk file id = 0

 

         Storage 1:

                   id = 192.168.8.202

                   ip_addr = 192.168.8.202  ACTIVE

                   http domain =

                   version = 5.10

                   join time = 2017-07-20 09:58:37

                   up time = 2017-07-20 09:58:37

                   total storage = 26917 MB

                   free storage = 21646 MB

                   upload priority = 10

                   store_path_count = 1

                   subdir_count_per_path = 256

                   storage_port = 23000

                   storage_http_port = 8888

                   current_write_path = 0

                   source storage id =

                   if_trunk_server = 0

                   connection.alloc_count = 256

                   connection.current_count = 0

                   connection.max_count = 0

                   total_upload_count = 0

                   success_upload_count = 0

                   total_append_count = 0

                   success_append_count = 0

                   total_modify_count = 0

                   success_modify_count = 0

                   total_truncate_count = 0

                   success_truncate_count = 0

                   total_set_meta_count = 0

                   success_set_meta_count = 0

                   total_delete_count = 0

                   success_delete_count = 0

                   total_download_count = 0

                   success_download_count = 0

                   total_get_meta_count = 0

                   success_get_meta_count = 0

                   total_create_link_count = 0

                   success_create_link_count = 0

                   total_delete_link_count = 0

                   success_delete_link_count = 0

                   total_upload_bytes = 0

                   success_upload_bytes = 0

                   total_append_bytes = 0

                  success_append_bytes = 0

                   total_modify_bytes = 0

                   success_modify_bytes = 0

                   stotal_download_bytes = 0

                   success_download_bytes = 0

                   total_sync_in_bytes = 0

                   success_sync_in_bytes = 0

                   total_sync_out_bytes = 0

                   success_sync_out_bytes = 0

                   total_file_open_count = 0

                   success_file_open_count = 0

                   total_file_read_count = 0

                   success_file_read_count = 0

                   total_file_write_count = 0

                   success_file_write_count = 0

                   last_heart_beat_time = 2017-07-20 10:03:15

                   last_source_update = 1969-12-31 16:00:00

                   last_sync_update = 1969-12-31 16:00:00

                   last_synced_timestamp = 1969-12-31 16:00:00

[root@localhost data]#

7

 

如上所示,看到192.168.8.202 ACTIVE 字样即可说明storage服务器已经成功登记到了tracker服务器,同理别忘了添加开机启动,打开/etc/rc.d/rc.local并将如下配置追加到文件中:

service fdfs_storaged start

至此我们就已经完成了fastdfs的全部配置,此时也就可以用客户端工具进行文件上传下载的测试了。

五、测试FastDFS

5.1配置 FastDFSLinux上的客户端

编辑配置文件/etc/fdfs/client.conf

命令:

[root@localhost fdfs]# vi /etc/fdfs/client.conf

修改内容:

  1. base_path=/opt/fastdfs_tracker #tracker服务器文件路径
  2. tracker_server=192.168.8.202:22122 #tracker服务器IP地址和端口号
  3. http.tracker_server_port=8800 # tracker 服务器的 http 端口号,必须和tracker的设置对应起来

8

9

10

 

5.2测试

配置完成后就可以模拟文件上传了,先给/opt目录下放一张图片(使用ftp类似工具上传到服务器)

11

[root@localhost fdfs]# ll /opt/

total 24

drwxr-xr-x. 4 root root 4096 Jul 20 09:58 fastdfs_storage

drwxr-xr-x. 3 root root 4096 Jul 20 09:58 fastdfs_storage_data

drwxr-xr-x. 4 root root 4096 Jul 20 09:44 fastdfs_tracker

drwxr-xr-x. 2 root root 4096 Nov 22  2013 rh

-rw-r--r--. 1 root root 7014 Jul 21 02:20 tx.jpg

[root@localhost fdfs]#

 

通过执行客户端上传命令尝试上传:

[root@localhost fdfs]# /usr/bin/fdfs_upload_file  /etc/fdfs/client.conf /opt/tx.jpg

group1/M00/00/00/wKgIyllxx82AeS-3AAAbZoAXENE710.jpg

[root@localhost fdfs]#

12

返回信息解释说明:

group1                                                                    : 组名

MOO                                                                         : 磁盘

00/00                                                                        : 目录(相对于storagefastdfs_storage_data目录下)

wKgIyllxx82AeS-3AAAbZoAXENE710.jpg     :文件名

 

可以切换到storeagedata目录下查看

命令:

[root@localhost data]# cd /opt/fastdfs_storage_data/data/

[root@localhost data]# ls

00  04  08  0C  10  14  18  1C  20  24  28  2C  30  34  38  3C  40  44  48  4C  50  54  58  5C  60  64  68  6C  70  74  78  7C  80  84  88  8C  90  94  98  9C  A0  A4  A8  AC  B0  B4  B8  BC  C0  C4  C8  CC  D0  D4  D8  DC  E0  E4  E8  EC  F0  F4  F8  FC

01  05  09  0D  11  15  19  1D  21  25  29  2D  31  35  39  3D  41  45  49  4D  51  55  59  5D  61  65  69  6D  71  75  79  7D  81  85  89  8D  91  95  99  9D  A1  A5  A9  AD  B1  B5  B9  BD  C1  C5  C9  CD  D1  D5  D9  DD  E1  E5  E9  ED  F1  F5  F9  FD

02  06  0A  0E  12  16  1A  1E  22  26  2A  2E  32  36  3A  3E  42  46  4A  4E  52  56  5A  5E  62  66  6A  6E  72  76  7A  7E  82  86  8A  8E  92  96  9A  9E  A2  A6  AA  AE  B2  B6  BA  BE  C2  C6  CA  CE  D2  D6  DA  DE  E2  E6  EA  EE  F2  F6  FA  FE

03  07  0B  0F  13  17  1B  1F  23  27  2B  2F  33  37  3B  3F  43  47  4B  4F  53  57  5B  5F  63  67  6B  6F  73  77  7B  7F  83  87  8B  8F  93  97  9B  9F  A3  A7  AB  AF  B3  B7  BB  BF  C3  C7  CB  CF  D3  D7  DB  DF  E3  E7  EB  EF  F3  F7  FB  FF

[root@localhost data]# cd 00/00/

[root@localhost 00]# ls

wKgIyllxx82AeS-3AAAbZoAXENE710.jpg

[root@localhost 00]#

13

测试完成。

相关推荐:
FastDFS卸载(二)
FastDFS配置集群(三)

地址:https://www.leftso.com/article/235.html

相关阅读

FastDFS分布式文件系统的下载和安装配置,FastDFS,FastDFS安装配置,分布式文件系统
FastDFS配置文件详细说明
FastDFS5.10卸载,在centos6系统中卸载FastDFS(二),FastDFS卸载,FastDFS
FastDFS集群配置(三)
Java编程之spring boot FastDFS Java client使用,Java编程,FastDFS Java客户端
linux系统之centos 7 安装和配置redis,redis是一个高性能的key-value数据库,通常用于分布式缓存
在Linux系统中安装和配置Java的运行或者编译jdk环境。安装分两种类型,系统安装包例如centos系列的有rpm包或者通过yum安装;第二总就是Linux通用的方法,下载对应系统的jdk压...
现在zookeeper在分布式上的流行程度不亚于spring在Java中的地位。本文主要讲解zookeeper的入门,即:安装zookeeper以及配置zookeeper集群