08 五种I/O模型&select模型

day08

本章目标

  • 五种I/O模型
  • select
  • select改进回射客户端程序

五种I/O模型

  • 阻塞I/O
  • 非阻塞I/O
  • I/O复用(selectpoll)
  • 信号驱动I/O
  • 异步I/O

1、阻塞I/O模型

阻塞等待数据到来。

阻塞IO模型

2、非阻塞I/O模型

**忙等待**,用的很少,程序一直在等待数据,没有数据相当于死循环,占用CPU资源。

非阻塞IO模型

fcntl(fd, F_SETFL, flag|O_NONBLOCK);

详情查看本博客C语言部分:跳转连接

PS:EMOULDBLOCK错误应当改为EWOULDBLOCK

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
NAME
fcntl - manipulate file descriptor

SYNOPSIS
#include <unistd.h>
#include <fcntl.h>

int fcntl(int fd, int cmd, ... /* arg */ );

DESCRIPTION
fcntl() performs one of the operations described below on the open file
descriptor fd. The operation is determined by cmd.
fcntl() can take an optional third argument. Whether or not this argu‐
ment is required is determined by cmd. The required argument type is
indicated in parentheses after each cmd name (in most cases, the
required type is int, and we identify the argument using the name arg),
or void is specified if the argument is not required.

RETURN VALUE
For a successful call, the return value depends on the operation:
F_DUPFD The new descriptor.
F_GETFD Value of file descriptor flags.
F_GETFL Value of file status flags.
F_GETLEASE
Type of lease held on file descriptor.
F_GETOWN Value of descriptor owner.
F_GETSIG Value of signal sent when read or write becomes possible, or
zero for traditional SIGIO behavior.
F_GETPIPE_SZ
The pipe capacity.
All other commands
Zero.
On error, -1 is returned, and errno is set appropriately.

3、I/O复用

IO复用模型

通过select来实现,通过select管理多个文件描述符。一旦由文件描述符监测到数据到来select就返回,然后调用recv函数对数据处理,

4、信号驱动I/O

不常用,非阻塞模式,有消息到来通过信号跳到消息处理函数,没有消息,处理其他事情。

信号驱动IO

5、异步I/O

效率最高的处理方式aio_read函数,有一个缓冲区,如果没有数据立即返回,有数据的话,把数据拷贝到应用层的缓冲区。复制完成后,通过信号通知应用层的数据。与第四种模型相似,但是又有很大区别。

异步IO

select模型 —>重点

man 2 select,学会使用帮助手册。

select:用来管理fd,地位:管理者,管理多个I/O,一旦其中的一个I/O监测到所感兴趣的事件,那么select函数返回,返回值为监测到事件的个数。并且返回那些I/O发生了那些事件。遍历事件然后进行处理,单进程处理比较好。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#include <sys/select.h>

/* According to earlier standards */
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

void FD_CLR(int fd, fd_set *set);
int FD_ISSET(int fd, fd_set *set);
void FD_SET(int fd, fd_set *set);
void FD_ZERO(fd_set *set);

#include <sys/select.h>

int pselect(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, const struct timespec *timeout, const sigset_t *sigmask);
  • select的参数说明:
    • int nfds:存放到读、写、异常的集合文件描述符的最大值+1。就是改参数的值。

    • fd_set *readfds:读的集合,如果有数据可读的套接口放在此集合,放的为文件描述符。输入输出型参数。

    • fd_set *writefds:写的集合,如果有数据可写的套接口放在此集合,放的为文件描述符。输入输出型参数。

    • fd_set *exceptfds:异常的集合输入输出型参数。

    • struct timeval *timeout:指定超时时间,NULL不设置超时时间。设置时间后如果没有监测到时间,也会返回返回值为0输入输出型参数。

      select函数每次返回一次,都需要调用FD_ZEROFD_SET,因为第二、三、四个参数都是输入输出型参数,返回后其值会发生变化,需要重新设置内容。

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      fd_set rset;
      FD_ZERO(&rset);
      int maxfd;
      int nready;
      char sendbuf[1024] = {0};
      char recvbuf[1024] = {0};

      int fd_stdin = fileno(stdin);//为了防止标准输入和输出被重定向所以用fileno转换stdin/stdout为fd
      if(fd_stdin > sock)//设置最大fd
      maxfd = fd_stdin;
      else
      maxfd = sock;
      while(1)
      {
      FD_SET(fd_stdin, &rset);//每次都需要添加在while循环,详情查看博客原因
      FD_SET(sock, &rset);
      nready = select(maxfd + 1, &rset, NULL, NULL, NULL);
      if(nready == -1)
      {
      ERR_EXIT("select");
      }
      else if(nready == 0)
      continue;
      if(FD_ISSET(sock, &rset))//判断是那个信号的,调其处理的函数
      {
      int ret = readline(sock, recvbuf, sizeof(recvbuf));
      if(ret == -1)
      {
      ERR_EXIT("readline");
      }
      else if(ret == 0)
      {
      printf("server close\n");
      break;
      }
      fputs(recvbuf, stdout);
      memset(recvbuf, 0, sizeof(recvbuf));
      }
      else if(FD_ISSET(fd_stdin, &rset))
      {
      if(fgets(sendbuf, sizeof(sendbuf), stdin) == NULL)
      break;
      writen(sock, sendbuf, strlen(sendbuf));
      memset(sendbuf, 0, sizeof(sendbuf));
      }
      }
      close(sock);
      return 0;
  • timeout的结构体:
1
2
3
4
5
6
7
8
9
10
11
12
The timeout
The time structures involved are defined in <sys/time.h> and look like
struct timeval {
long tv_sec; /* seconds */
long tv_usec; /* microseconds */
};
and
struct timespec {
long tv_sec; /* seconds */
long tv_nsec; /* nanoseconds */
};
> (However, see below on the POSIX.1-2001 versions.)
  • select的返回值:
1
2
RETURN VALUE
On success, select() and pselect() return the number of file descriptors contained in the three returned descriptor sets (that is, the total number of bits that are set in readfds, writefds, exceptfds) which may be zero if the timeout expires before anything interesting happens. On error, -1 is returned, and errno is set appropriately; the sets and timeout become undefined, so do not rely on their contents after an error.
  • 四个宏

    • void FD_CLR(int fd, fd_set *set);

      如果fd存在于集合中,将fd从集合中移除。

    • int FD_ISSET(int fd, fd_set *set);

      判定fd是否再集合中,不会改变集合的内容。

    • void FD_SET(int fd, fd_set *set);

      fd添加到集合当中

    • void FD_ZERO(fd_set *set);

      将集合清空,即删除集合内所有的fd

10 close和shutdown函数

day10

本章目标

  • closeshutdown的区别
  • 进一步改进回射客户端程序

close和shutdown

  • close终止了数据传送的两个方向

  • shutdown可以有选择的终止某个方向的数据传送或者终止数据传送的两个方向

  • shutdown how=1就可以保证对等方接收到一个EOF字符,而不管其他进程是否已经打开了套接字,即不关心引用计数。而close不能保证一定给对等方发送EOF,而是直到套接字引用计数减为0时才发送,也就是说直到所有的进程都关闭了套接字。才是真的关闭了套接字。

    FIN:是用来扫描保留的端口,发送一个FIN包(或者是任何没有ACKSYN标记的包)到目标的一个开放的端口,然后等待回应。许多系统会返回一个复位标记。

当客户端(或者服务器)发送消息后,另一端没有读取消息,消息还在管道中,但是客户端(或者服务器)使用close关闭了套接字,那么另一端就不能把未读取的数据读取出来。所以需要shutdown,终止某个方向的数据传送。

关于shutdownclose参考文档:https://blog.csdn.net/lgp88/article/details/7176509

shutdown

man 2 shutdown查看帮助,这个是一个函数,和linux命令一样但是… …,这俩不一样

假设客户端使用了close(fd),那么客户端的其他进程如果也使用了这个套接字(即:引用计数 > 1),则只是引用计数减一操作,其他的进程仍然可以通过这个套接字与服务器进行通信。

假设客户端如果使用了shutdown(fd, SHUT_RD),则直接关闭读端(不管引用计数为多少),其他进程则不能通过读端再进行数据的读取了,同理可得SHUT_WR:关闭写端,SHUT_RDWR:读端和写端都关闭。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
#include <sys/socket.h>

int shutdown(int sockfd, int how);

DESCRIPTION
The shutdown() call causes all or part of a full-duplex connection on the socket associated with sockfd to be shut down. If how is SHUT_RD, further receptions will be disallowed. If how is SHUT_WR, further transmissions will be disallowed. If how is SHUT_RDWR, further receptions and transmissions will be disallowed.

RETURN VALUE
On success, zero is returned. On error, -1 is returned, and errno is set appropriately.

ERRORS
EBADF
sockfd is not a valid descriptor.
EINVAL
An invalid value was specified in how (but see BUGS).
ENOTCONN
The specified socket is not connected.
ENOTSOCK
sockfd is a file, not a socket.
  • 参数一:shutdown的套接字描述符
  • 参数二:关闭的状态:TCP/IP是全双工的
    • SHUT_RD:关闭读端,宏定义为0
    • SHUT_WR:关闭写端,宏定义为1
    • SHUT_RDWR:读端和写端都关闭,宏定义为2

注意啦

本篇博客请重点查看下代码,因为这篇博客不好整理,所以… …

echocli.c185-186行205行215-217行

echosrv.c279-282行286行

有注释所以应该可以比较好理解

09 select并发服务器

day09

本章目标

  • select
  • 读、写、异常事件发生条件
  • select改进回射服务器程序

select

再复习和补充一点select的参数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#include <sys/select.h>

/* According to earlier standards */
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

void FD_CLR(int fd, fd_set *set);
int FD_ISSET(int fd, fd_set *set);
void FD_SET(int fd, fd_set *set);
void FD_ZERO(fd_set *set);

#include <sys/select.h>

int pselect(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, const struct timespec *timeout, const sigset_t *sigmask);
  • 参数一:int nfds

    因为select的对感兴趣的fd取值范围为[0, nfds),前闭后开区间,所以需要最大的文件描述符+1,才能覆盖整个感兴趣的文件描述符号的范围。

  • 参数二:fd_set *readfds

    可读事件集合,添加感兴趣可读fd

  • 参数三:fd_set *writefds

    可写事件集合,添加感兴趣可写fd

  • 参数四:fd_set *exceptfds

    异常事件集合

  • 参数五:struct timeval *timeout

    超时时间,在响应的超时时间内没有发生任何一个I/O事件(读/写事件),则select返回0,指示超时。出现错误select返回-1.

监测到多个事件,需要遍历,readfdswritefdsexceptfds这些集合来一一处理这些事情,则将select实现的服务器称为并发服务器。因为没有能力并行处理这些I/O事件,只能一一遍历处理这些事件。所以称为并发服务器,而不是并行服务器。并发服务器不能长时间处理这些事件,因为长时间处理则不能去执行其他的动作。且事件间的处理间隔时间也不能太长。无法充分利用多核处理器。使用多进程select或者多线程处理这些事件。

select可以同时检测网络I/O和标准I/O事件。那个事件产生则处理那个事件。不会因为某一个阻塞导致另一个无法处理。

读、写、异常事件发生条件

  • 可读
    • 套接口缓冲区有数据可读,底层本质:如果缓冲区的数据超过一定的范围才会通知有数据可读
    • 连接的读一般关闭,即接收到FIN段,读操作返回0
    • 如果是监听套接口,已完成连接队列不为空时。
    • 套接口上发生了一个错误待处理,错误可以通过getsockopt指定SO_ERROR选项来获取
  • 可写
    • 套接口发送缓冲区有空间容纳数据
    • 连接的写一半关闭,即收到RST段之后,再次调用write操作
    • 套接口上发生一个错误待处理,错误可以通过getsockopt指定SO_ERROR选项来获取
  • 异常
    • 套接口存在带外数据。

并发服务器

对上一次的服务器端通过select使用一个进程实现。并发服务器的连接客户端的限制,FD_SETSZIE大小和一个进程当中能够打开的I/O的最大数目。FD_SETSIZE不能被更改(非要更改,需要更改内核代码,并进行重新编译),而一个进程的可以打开的最大的I/O数目可以更改,通过ulimit -n 1000命令更改一个进程可以打开的最大I/O数目为1000FD_SETSIZE默认情况下和进程能够打开的最大I/O数目为1024个。

核心代码(部分代码)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
int i = 0;
int num = 0;
int maxi = 0; //最大不空闲的套接口下标
int nready = 0;
int maxfd = listenfd;
int conn[FD_SETSIZE] = {0};
fd_set rset;
fd_set allset;
FD_ZERO(&rset);
FD_ZERO(&allset);
FD_SET(listenfd, &allset);
struct sockaddr_in sockname;
socklen_t addrlen = sizeof(sockname);//一定要有初始值,为结构体大小,输入输出型参数。
for(i = 0; i < FD_SETSIZE; i++)//初始化conn数组
{
conn[i] = -1;
}
while(1)//一个进程处理并发
{
rset = allset;
nready = select(maxfd + 1, &rset, NULL, NULL, NULL);
if(nready == -1)
{
//errno 是记录系统的最后一次错误代码。
//EINTR错误的产生:当阻塞于某个慢系统调用的一个进程捕获某个信号且相应信号处理函数返回时,该系统调用可能返回一个EINTR错误。
if(errno == EINTR)//如果被信号中断则继续
continue;
ERR_EXIT("select");//其他错误退出
}
else if(nready == 0)
{
//时间超时,因为这里没有设置超时时间,理论不会出现nready == 0的情况
continue;
}
if(FD_ISSET(listenfd, &rset))//判断监听套接口是否在集合当中,处理监听套接口的数据
{
int ret = -1;
if((ret = accept(listenfd, (struct sockaddr *)&peeraddr, (socklen_t *)&peerlen)) < 0)
{
ERR_EXIT("accept");
}
for(i = 0; i < FD_SETSIZE; i++)
{
if(conn[i] < 0)
{
conn[i] = ret;
if(i > maxi)
maxi = i;//最大不空闲套接口下标,优化遍历时间,提高效率
break;
}
}
if(i == FD_SETSIZE)
{
//stderr标准错误
fprintf(stderr, "链接的客户端太多了... ...");
exit(EXIT_FAILURE);
}
//获取远程的地址
if(0 > getpeername(ret , (struct sockaddr *)&sockname, (socklen_t*)&addrlen))
{
perror("getsockname");
exit(EXIT_FAILURE);//退出程序
}
printf("远程的IP: %s, Port: %d\n", inet_ntoa(sockname.sin_addr), ntohs(sockname.sin_port));
FD_SET(ret, &allset);//添加新的套接口到allset中下一次select会检测这个套接口
if(maxfd < ret)//需要更新最大描述符
maxfd = ret;
if(--nready <= 0)//已经处理完了监听套接口的事件,把nready减一操作,如果等于零,说明没有要处理的事件了,为了提高效率直接continue,如果不小于等于零,说明还有其他事件,即为连接客户端的事件,>然后对客户端的事件进行处理,
continue;
}
for(i = 0; i <= maxi; i++)
{
if(conn[i] == -1)
continue;
if(FD_ISSET(conn[i], &rset))//判断是否在读的集合当中
{
//读取数据
char recvbuf[1024] = {0};
int ret = readline(conn[i], recvbuf, 1024);
if(ret == -1)
{
ERR_EXIT("readline");
}
if(0 == ret)
{
printf("对方关闭了Socket readline\n");
FD_CLR(conn[i], &allset);//从allset中清除
conn[i] = -1;//标志为-1
continue;
}
printf("服务器接收到的数据:");
fputs(recvbuf, stdout);
writen(conn[i], recvbuf, strlen(recvbuf));//数据回射回去:
if(--nready <= 0)
break;
}
}
FD_ZERO(&rset);//清空rset
}

11 套接字IO超时设置&errno&fcntl&getsockopt

day11

本章目标

  • 套接字I/O超时设置方法
  • select实现超时
    • read_timeout函数封装
    • write_timeout函数封装
    • accept_timeout函数封装
    • connect_timeout函数封装

套接字I/O超时设置方法

一共有三种方法

  • alarm:闹钟定时
  • 套接字选项:移植性比较差
    • SO_SNDTIMEO
    • SO_RCVTIMEO
  • select

用select实现超时

自定义read_timeoutwrite_timeoutaccept_timeoutconnect_timeout函数对select函数进行了封装,所需要参数为文件描述符时间长度

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

The timeout
The time structures involved are defined in <sys/time.h> and look like

   struct timeval {
       long    tv_sec;         /* seconds */
       long    tv_usec;        /* microseconds */
   };

and

   struct timespec {
       long    tv_sec;         /* seconds */
       long    tv_nsec;        /* nanoseconds */
   };

##errno.h头文件

里面设置了很多的错误信息,可以通过错误信息来对函数的成功与否,以及错误原因进行详细的接收。

参考文档:

http://blog.163.com/wangxun_2233/blog/static/5511009120094795756775

这个博客写的很详细,不过这个没必须记,会查就行了

getsockopt函数

参考文档:

https://blog.csdn.net/daiyudong2020/article/details/51893399

level:设置为SOL_SOCKETIPPROTO_TCP,详情参考上面的链接,一般设置为SOL_SOCKET

optname:参数SO_ERROR获取待处理错误并清除。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
NAME
getsockopt, setsockopt - get and set options on sockets

SYNOPSIS
#include <sys/types.h> /* See NOTES */
#include <sys/socket.h>

int getsockopt(int sockfd, int level, int optname,
void *optval, socklen_t *optlen);
int setsockopt(int sockfd, int level, int optname,
const void *optval, socklen_t optlen);

DESCRIPTION
getsockopt() and setsockopt() manipulate options for the socket referred to by the file descriptor sockfd. Options may exist at multiple protocol levels; they are always present at the uppermost socket level.
When manipulating socket options, the level at which the option resides and the name of the option must be specified. To manipulate options at the sockets API level, level is specified as SOL_SOCKET. To manipulate options at any other level the protocol number of the appropriate protocol controlling the option is supplied. For example, to indicate that an option is to be interpreted by the TCP protocol, level should be set to the protocol number of TCP; see getprotoent(3).
The arguments optval and optlen are used to access option values for setsockopt(). For getsockopt() they identify a buffer in which the value for the requested option(s) are to be returned. For getsockopt(), optlen is a valueresult argument, initially containing the size of the buffer pointed to by optval, and modified on return to indicate the actual size of the value returned. If no option value is to be supplied or returned, optval may be NULL.
Optname and any specified options are passed uninterpreted to the appropriate protocol module for interpretation. The include file <sys/socket.h> contains definitions for socket level options, described below. Options at other protocol levels vary in format and name; consult the appropriate entries in section 4 of the manual.
Most socket-level options utilize an int argument for optval. For setsockopt(), the argument should be nonzero to enable a boolean option, or zero if the option is to be disabled.
For a description of the available socket options see socket(7) and the appropriate protocol man pages.

RETURN VALUE
On success, zero is returned. On error, -1 is returned, and errno is set appropriately.

一个函数fcntl

参考文档:

https://blog.csdn.net/fengxinlinux/article/details/51980837

fcntl函数,man 3 fcntlfcntl可实现对指定文件描述符的各种操作

以下为该函数的man手册的说明,如果看不懂请到上面的连接中去学习该函数,我觉得上面的博客已经写的还不错吧,而且我的英语不太好,所以我就不卖弄自己垃圾的英语了… …。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
ROLOG
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ
(consult the corresponding Linux manual page for details of Linux behavior), or the interface may not be implemented
on Linux.

NAME
fcntl - file control

SYNOPSIS
#include <unistd.h>
#include <fcntl.h>

int fcntl(int fildes, int cmd, ...);

DESCRIPTION
The fcntl() function shall perform the operations described below on open files. The fildes argument is a file
descriptor.

The available values for cmd are defined in <fcntl.h> and are as follows:

F_DUPFD//复制文件描述符
Return a new file descriptor which shall be the lowest numbered available (that is, not already open) file descriptor greater than or equal to the third argument, arg, taken as an integer of type int. The new file descriptor shall refer to the same open file description as the original file descriptor, and shall share any locks. The FD_CLOEXEC flag associated with the new file descriptor shall be cleared to keep the file open across calls to one of the exec functions.

F_GETFD//获取文件描述符
Get the file descriptor flags defined in <fcntl.h> that are associated with the file descriptor fildes. File descriptor flags are associated with a single file descriptor and do not affect other file descriptors that refer to the same file.

F_SETFD//设置文件描述符
Set the file descriptor flags defined in <fcntl.h>, that are associated with fildes, to the third argument, arg, taken as type int. If the FD_CLOEXEC flag in the third argument is 0, the file shall remain open across the exec functions; otherwise, the file shall be closed upon successful execution of one of the exec func‐ tions.

F_GETFL//获取文件状态标志
Get the file status flags and file access modes, defined in <fcntl.h>, for the file description associated with fildes. The file access modes can be extracted from the return value using the mask O_ACCMODE, which is defined in <fcntl.h>. File status flags and file access modes are associated with the file description and do not affect other file descriptors that refer to the same file with different open file descriptions.

F_SETFL//设置文件状态标志
Set the file status flags, defined in <fcntl.h>, for the file description associated with fildes from the corresponding bits in the third argument, arg, taken as type int. Bits corresponding to the file access mode and the file creation flags, as defined in <fcntl.h>, that are set in arg shall be ignored. If any bits in arg other than those mentioned here are changed by the application, the result is unspecified.

F_GETOWN//获取当前接收SIGIO和SIGURG信号的进程嗯IO和进程组ID
If fildes refers to a socket, get the process or process group ID specified to receive SIGURG signals when out-of-band data is available. Positive values indicate a process ID; negative values, other than -1, indi‐cate a process group ID. If fildes does not refer to a socket, the results are unspecified.

F_SETOWN//设置当前接收SIGIO和SIGURG信号的进程嗯IO和进程组ID
If fildes refers to a socket, set the process or process group ID specified to receive SIGURG signals when out-of-band data is available, using the value of the third argument, arg, taken as type int. Positive values indicate a process ID; negative values, other than -1, indicate a process group ID. If fildes does not refer to a socket, the results are unspecified.

The following values for cmd are available for advisory record locking. Record locking shall be supported for regular files, and may be supported for other files.

F_GETLK//获取文件锁
Get the first lock which blocks the lock description pointed to by the third argument, arg, taken as a pointer to type struct flock, defined in <fcntl.h>. The information retrieved shall overwrite the information passed to fcntl() in the structure flock. If no lock is found that would prevent this lock from being created, then the structure shall be left unchanged except for the lock type which shall be set to F_UNLCK.

F_SETLK//设置文件锁
Set or clear a file segment lock according to the lock description pointed to by the third argument, arg, taken as a pointer to type struct flock, defined in <fcntl.h>. F_SETLK can establish shared (or read) locks (F_RDLCK) or exclusive (or write) locks (F_WRLCK), as well as to remove either type of lock (F_UNLCK). F_RDLCK, F_WRLCK, and F_UNLCK are defined in <fcntl.h>. If a shared or exclusive lock cannot be set, fcntl() shall return immediately with a return value of -1.

F_SETLKW//与F_SETLK设置文件锁类似,不过等待返回
This command shall be equivalent to F_SETLK except that if a shared or exclusive lock is blocked by other locks, the thread shall wait until the request can be satisfied. If a signal that is to be caught is received while fcntl() is waiting for a region, fcntl() shall be interrupted. Upon return from the signal handler, fcntl() shall return -1 with errno set to [EINTR], and the lock operation shall not be done.

read_timeout函数封装

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
int read_timeout(int fd, unsinged int wait_seconds)
{
int ret = 0;
if(wait_seconds > 0)
{
fd_set rset;
FD_ZERO(&rset);
FD_SET(fd, &rset);//把fd添加到集合当中
struct timeval time;
time.tv_sec = wait_seconds;
time_tv_usec = 0;
do
{ //超时检测,时间到了,却未发生可读事件,返回0
ret = select(fd + 1, &rset, NULL, NULL, &timeout);
}while(ret < 0 && errno == EINTR);//中断后重启
//如果ret < 0 则说明select函数是被中断的
//errno是记录程序的最后一次错误代码
//EINTR:当阻塞于某个慢系统调用的一个进程捕获某个信号且相应信号处理函数返回时,该系统调用可能返回一个EINTR错误。(查看网络编程的day09)
if(ret == 0)
{
ret = -1;
errno = ETIMEOUT;
}
else if(ret == 1)
{
ret = 1;
}
}
return ret;
}

剩下三个函数的封装

剩下的三个函数的封装在下载文件中,write_timeoutread_timeout类似,其他两个有所不同

点击下载源文件

12 select函数&getrlimit&setrlimit&poll函数

day12

本章目标

  • select限制
  • poll,也是I/O复用函数

select限制

  • select实现的并发服务器,能达到的并发数,受两方面的限制

    • 一个进程所能打开的最大文件描述符限制,可以通过调整内核参数来改变,ulimit -n查看能打开的文件描描述符的大小,通过ulimit -n num:通过这条命令修改一个文件所能打开的最大文件描述符的限制,num为改变之后的最大文件描述符。这个命令只是临时修改,如果需要开机的时候改变的话则需要修改配置文件。

      发现几个ulimit的小坑,以下皆为个人猜想,有一定的实践操作佐证。

      ulimit -n num:如果是普通用户每次修改的文件描述符的num必须小于ulimit -n的数值的大小,否则会报错误bash: ulimit: open files: cannot modify limit: Operation not permitted,权限不够,emmm。如果是管理员则不会出现这个错误,可以修改成功。

      即:管理员可以随意的改大改小,而普通用户只能不断的改小,不能改的比原来的最大能打开的文件描述符数量大。但是可以越改越小… …。

      今天下午又试了一次发现普通用户也可以把最大能打开的文件描述符数量增大。… ….

    • select中的fd_set集合容量的限制(FD_SETSIZE,一般为1024),这个需要重新编译内核。

这里是一条插播的关于vim内容

参考文档:

https://baike.1688.com/doc/view-d36947406.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
TAB就是制表符,单独拿出来做一节是因为这个东西确实很有用.
> 输入此命令则光标所在行向右移动一个tab.
5>> 输入此命令则光标后5行向右移动一个tab.
:12,24> 此命令将12行到14行的数据都向右移动一个tab.
:12,24>> 此命令将12行到14行的数据都向右移动两个tab.
那么如何定义tab的大小呢?有人愿意使用8个空格位,有人用4个,有的用2个.
有的人希望tab完全用空格代替,也有的人希望tab就是tab.没关系,vim能
帮助您.以下的配置一般也都先写入配置文档中,免得老敲.
:set shiftwidth=4 配置自动缩进4个空格,当然要设自动缩进先.
:set sts=4 即配置softtabstop为4.输入tab后就跳了4格.
:set tabstop=4 实际的tab即为4个空格,而不是缺省的8个.
:set expandtab 在输入tab后,vim用恰当的空格来填充这个tab.
命令模式
dG从前行一直删除到文件末尾。
u:撤销操作,类似win的ctrl + z
ctrl + r:重做上一个操作

getrlimit和setrlimit

获取一个进程所能打开的最大文件描述符数目。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
#include <sys/time.h>
#include <sys/resource.h>

int getrlimit(int resource, struct rlimit *rlim);
int setrlimit(int resource, const struct rlimit *rlim);
int prlimit(pid_t pid, int resource, const struct rlimit *new_limit, struct rlimit *old_limit);

resource:这里是获取进程能够打开的最大文件描述符个数,所以第一个参数为,RLIMIT_NOFILE。
RLIMIT_NOFILE
Specifies a value one greater than the maximum file descriptor number that can be opened by this process. Attempts (open(2), pipe(2), dup(2), etc.) to exceed this limit yield the error EMFILE. (Historically, this limit was named RLIMIT_OFILE on BSD.)

第二个参数的结构体:使用时自己不需要再定义了。
struct rlimit {
rlim_t rlim_cur; /* Soft limit */
rlim_t rlim_max; /* Hard limit (ceiling for rlim_cur) */
};

RETURN VALUE
On success, these system calls return 0. On error, -1 is returned, and errno is set appropriately.

getrlimit和setrlimit函数使用代码

如果提示权限不够则使用管理员运行。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h> /* See NOTES */
#include <sys/socket.h>
#include <unistd.h>
#include <errno.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <arpa/inet.h>
#include <string.h>
#include <signal.h>
#include <sys/time.h>
#include <sys/resource.h>

#define ERR_EXIT(err) {perror(err); exit(EXIT_FAILURE);}


int main()
{
struct rlimit limit;
if(-1 == getrlimit(RLIMIT_NOFILE, &limit))//获取本进程limit信息
{
ERR_EXIT("getrlimit");
}
printf("old: current = %lld max = %lld\n", limit.rlim_cur, limit.rlim_max);

limit.rlim_cur = 2048;
limit.rlim_max = 2048;

if(-1 == setrlimit(RLIMIT_NOFILE, &limit))//重新设置
{
ERR_EXIT("setrlimit");
}
if(-1 == getrlimit(RLIMIT_NOFILE, &limit))//获取修改后的信息
{
ERR_EXIT("getrlimit");
}
printf("new: current = %lld max = %lld\n", limit.rlim_cur, limit.rlim_max);

return 0;
}

测试一个进程可以打开的最大文件文件描述符

实际上tag只能到1021(ulimit 设置为1024时)。

why???,因为还有三个系统已经给你打开的标准输入0标准输出1标准错误2。加起来刚好1024个。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h> /* See NOTES */
#include <sys/socket.h>
#include <unistd.h>
#include <errno.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <string.h>
#include <signal.h>
#include <sys/select.h>

#define ERR_EXIT(err) {perror(err); exit(EXIT_FAILURE);}

//只作连接操作
int main()
{
int tag = 0;
while(1)
{
int sock;
int i = 0;
struct sockaddr_in servaddr;//一个服务器的地址

sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
if(sock < 0)
{
perror("创建scoket失败\n");
exit(EXIT_FAILURE);//退出程序
}
memset(&servaddr, 0, sizeof(servaddr));//初始化地址
servaddr.sin_family = AF_INET;//地址族
servaddr.sin_port = htons(6000);//端口号,网络字节序转化为主机字节序
servaddr.sin_addr.s_addr = inet_addr("127.0.0.1");//指定服务端的地址

int conn = -1;
if((conn = connect(sock, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0))
{
printf("连接服务器失败\n");
exit(EXIT_FAILURE);
}

//获取本地的地址
struct sockaddr_in sockname;
socklen_t addrlen = sizeof(sockname);
if(0 > getsockname(sock, (struct sockaddr *)&sockname, (socklen_t *)&addrlen))
{
perror("getsockname");
exit(EXIT_FAILURE);//退出程序
}
tag++;
printf("%d\n", tag);
}
return 0;
}

部分程序运行结果(客户端):这个结果有一定的问题,但也是正确的,请仔细阅读理解下面的对于这个原因的解释。

1
2
3
4
5
6
1019
1020
1021
创建scoket失败
: Too many open files
[pip@localhost code]$

服务器端理论应该为:1020个,因为不仅有标准输入0标准输出1标准错误2,还有一个监听套接listenfd占用了一个文件描述符。则最多接受1020个连接了。如果服务端一直保持不断开的话。那么客户端也只能连接1020个,但是由于服务器与客户端的链接关闭了,导致可以连接客户端多于1020个了。所以上面的客户端的运行结果有一定的问题,服务器的对客户端处理合适,客户端的最大连接到服务器的个数为1020个,短板效应。

poll函数

也是并发服务器的函数,与select类似

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#include <poll.h>

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

struct pollfd {
int fd; /* file descriptor */
short events; /* requested events */
short revents; /* returned events */
};
结构体的成员变量events:
The bits that may be set/returned in events and revents are defined in <poll.h>:

POLLIN //有数据可读时
There is data to read.
POLLPRI
There is urgent data to read (e.g., out-of-band data on TCP socket; pseudoterminal master in packet mode has seen state change in slave).

POLLOUT
Writing now will not block.

POLLRDHUP (since Linux 2.6.17)
Stream socket peer closed connection, or shut down writing half of connection. The _GNU_SOURCE feature test macro must be defined (before including any header files) in order to obtain this definition.

POLLERR
Error condition (output only).

POLLHUP
Hang up (output only).

POLLNVAL
Invalid request: fd not open (output only).

##struct pollfd的events附录

poll_events

14 UDP服务器

day14

本章目标

  • UDP特点
  • UDP客户/服务基本模型
  • UDP回射客户/服务器
  • UDP注意点

UDP特点

  • 无连接
  • 基于消息的数据传输服务,理解为数据包之间有边界。
  • 不可靠
  • 一般情况下UDP更加高效

UDP注意点

  • UDP报文可能会丢失、重复
  • UDP报文可能会乱序
  • UDP缺乏流量控制
  • UDP协议数据报文截断
  • recvfrom返回为0,不代表连接关闭,因为UDP是无连接的
  • ICMP异步错误
  • UDP connect
  • UDP外出接口的确定

echosrv.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <string.h>

#define ERR_EXIT(x) do{perror(x); exit(EXIT_FAILURE);}while(0)

void echo_srv(int sock)
{
int ret = 0;
char recvbuf[1024] = {0};
struct sockaddr_in peeraddr;
socklen_t peerlen = sizeof(peeraddr);
while(1)
{
memset(recvbuf, 0, sizeof(recvbuf));
ret = recvfrom(sock, recvbuf, 1024, 0, (struct sockaddr *)&peeraddr, &peerlen);
if(ret < 0)
{
if(errno == EINTR)
continue;
ERR_EXIT("recvfrom");

}
else if(ret > 0)
{
fputs(recvbuf, stdout);
sendto(sock, recvbuf, strlen(recvbuf), 0, (struct sockaddr *)&peeraddr, peerlen);
}
}
close(sock);
}


int main()
{
int sock = 0;
sock = socket(PF_INET, SOCK_DGRAM, 0);//参数一表示为IPV4地址家族,第二个参数IPV4对应的UDP套接口,第三个参数0表示内核自动选择协议根据前两个可以确定通信协议为UDP的
if(sock < 0)
{
ERR_EXIT("socket");
}
struct sockaddr_in servaddr;
memset(&servaddr, 0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = 6000;
//servaddr.sin_addr = htonl("127.0.0.1");//和下面的一样作用,绑定指定的地址,和绑定任意一个地址。
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
if(bind(sock, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0)
ERR_EXIT("bind");
//不需要监听,直接连接的。
echo_srv(sock);
return 0;
}

echocli.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <string.h>

#define ERR_EXIT(x) do{perror(x); exit(EXIT_FAILURE);}while(0)

void echo_cli(int sock)
{
struct sockaddr_in servaddr;
memset(&servaddr, 0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = 6000;
servaddr.sin_addr.s_addr = inet_addr("127.0.0.1");//绑定服务器地址
if(connect(sock, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0)//连接后只能给指定的IP发送数据,则sendto可以不需要指定IP和端口了,UDP的connect是进行了一个弱绑定,不会进行三次握手和四次挥手
ERR_EXIT("connect");
int ret = 0;
char sendbuf[1024] = {0};
char recvbuf[1024] = {0};
while(1)
{
memset(sendbuf, 0, sizeof(sendbuf));
memset(recvbuf, 0, sizeof(recvbuf));
if(fgets(sendbuf, sizeof(sendbuf), stdin) == NULL)
ERR_EXIT("fgets");
//sendto(sock, sendbuf, strlen(sendbuf), 0, NULL, 0);
send(sock, sendbuf, strlen(sendbuf), 0);
ret = recvfrom(sock, recvbuf, sizeof(recvbuf), 0, NULL, NULL);
if(ret < 0)
{
if(errno == EINTR)
continue;
ERR_EXIT("recvbrom");
}
fputs(recvbuf, stdout);
}

}

int main()
{
int sock = 0;
sock = socket(PF_INET, SOCK_DGRAM, 0);//参数一表示为IPV4地址家族,第二个参数IPV4对应的UDP套接口,第三个参数0表示内核自动选择协议根据前两个可以确定通信协议为UDP的
if(sock < 0)
{
ERR_EXIT("socket");
}
echo_cli(sock);
return 0;
}

13 epoll相关函数、select、epoll和poll区别

day13

本章目标

  • epoll使用
  • epollselectpoll区别
  • epoll LR/ET模式

复习select和poll

  • select

    • 一个进程能打开的最大文件描述符个数是有限的
    • FD_SETSIZE的限制
  • poll

    • 一个进程能打开的最大文件描述符个数是有限的,系统能打开的最大文件描述符个数是有限的,与系统内存有关cat /proc/sys/fs/file-max,查看系统能打开的最大的文件描述符的个数。

      ulimit -n num

  • 共同点:

    内核需要遍历所有的文件描述符,直到找到发生事件的文件描述符。—>提高效率点,epoll对此进行了优化

epoll相关函数

  • int epoll_create(int size);//创建一个epoll实例,使用哈希表存储的,size是内部创建的哈希表大小
  • int epoll_create1(int flags);//同上,使用的是红黑数,所以不再需要指定大小了.—>优先选择使用.
  • int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);//把一个套接字添加到实例中,进行管理
  • int epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout);//等待事件

man 2 epoll_create

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
SYNOPSIS
#include <sys/epoll.h>

int epoll_create(int size);
int epoll_create1(int flags);

DESCRIPTION
epoll_create() creates an epoll(7) instance. Since Linux 2.6.8, the size argument is ignored, but must be greater than zero; see NOTES below.
epoll_create() returns a file descriptor referring to the new epoll instance. This file descriptor is used for all the subsequent calls to the epoll interface. When no longer required, the file descriptor returned by epoll_create() should be closed by using close(2). When all file descriptors referring to an epoll instance have been closed, the kernel destroys the instance and releases the associated resources for reuse.

epoll_create1()
If flags is 0, then, other than the fact that the obsolete size argument is dropped, epoll_create1() is the same as epoll_create(). The following value can be included in flags to obtain different behavior:

EPOLL_CLOEXEC
Set the close-on-exec (FD_CLOEXEC) flag on the new file descriptor. See the description of the O_CLOEXEC flag in open(2) for reasons why this may be useful.

RETURN VALUE
On success, these system calls return a nonnegative file descriptor. On error, -1 is returned, and errno is set to indicate the error.

man 2 epoll_ctl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
SYNOPSIS
#include <sys/epoll.h>

int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);

DESCRIPTION
This system call performs control operations on the epoll(7) instance referred to by the file descriptor epfd. It requests that the operation op be performed for the target file descriptor, fd.

Valid values for the op argument are :

EPOLL_CTL_ADD
Register the target file descriptor fd on the epoll instance referred to by the file descriptor epfd and associate the event event with the internal file linked to fd.

EPOLL_CTL_MOD
Change the event event associated with the target file descriptor fd.

EPOLL_CTL_DEL
Remove (deregister) the target file descriptor fd from the epoll instance referred to by epfd. The event is ignored and can be NULL (but see BUGS below). The event argument describes the object linked to the file descriptor fd. The struct epoll_event is defined as :

typedef union epoll_data {
void *ptr;
int fd;
uint32_t u32;
uint64_t u64;
} epoll_data_t;

struct epoll_event {
uint32_t events; /* Epoll events */
epoll_data_t data; /* User data variable */
};

//epoll_event成员events的参数
The events member is a bit set composed using the following available event types:

EPOLLIN
The associated file is available for read(2) operations.

EPOLLOUT
The associated file is available for write(2) operations.

EPOLLRDHUP (since Linux 2.6.17)
Stream socket peer closed connection, or shut down writing half of connection. (This flag is especially useful for writing simple code to detect peer shutdown when using Edge Triggered monitoring.)

EPOLLPRI
There is urgent data available for read(2) operations.

EPOLLERR
Error condition happened on the associated file descriptor. epoll_wait(2) will always wait for this event; it is not necessary to set it in events.

EPOLLHUP
Hang up happened on the associated file descriptor. epoll_wait(2) will always wait for this event; it is not necessary to set it in events.

EPOLLET
Sets the Edge Triggered behavior for the associated file descriptor. The default behavior for epoll is Level Triggered. See epoll(7) for more detailed information about Edge and Level Triggered event distribution architectures.

EPOLLONESHOT (since Linux 2.6.2)
Sets the one-shot behavior for the associated file descriptor. This means that after an event is pulled out with epoll_wait(2) the associated file descriptor is internally disabled and no other events will be reported by the epoll interface. The user must call epoll_ctl() with EPOLL_CTL_MOD to rearm the file descriptor with a new event mask.

man 2 epoll_wait

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
SYNOPSIS
#include <sys/epoll.h>

int epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout);
int epoll_pwait(int epfd, struct epoll_event *events,int maxevents, int timeout, const sigset_t *sigmask);

DESCRIPTION
The epoll_wait() system call waits for events on the epoll(7) instance referred to by the file descriptor epfd. The memory area pointed to by events will contain the events that will be available for the caller. Up to maxevents are returned by epoll_wait(). The maxevents argument must be greater than zero.

The timeout argument specifies the minimum number of milliseconds that epoll_wait() will block. (This interval will be rounded up to the system clock granularity, and kernel scheduling delays mean that the blocking interval may overrun by a small amount.) Specifying a timeout of -1 causes epoll_wait() to block indefinitely, while specifying a timeout equal to zero cause epoll_wait() to return immediately, even if no events are available.

The struct epoll_event is defined as :

typedef union epoll_data {
void *ptr;
int fd;
uint32_t u32;
uint64_t u64;
} epoll_data_t;

struct epoll_event {
uint32_t events; /* Epoll events */
epoll_data_t data; /* User data variable */
};

The data of each returned structure will contain the same data the user set with an epoll_ctl(2) (EPOLL_CTL_ADD,EPOLL_CTL_MOD) while the events member will contain the returned event bit field.

epoll实现并发服务器—>优

epoll实现服务器会占用五个套接字,标准输入标准输出标准错误服务器监听套接字epoll的句柄。剩下的,所以当进程的``最大套接字的个数 - 5,就是epoll可以连接的最大客户端个数了。epollpoll`效率要高。

点击查看epollsrv代码

epoll、select和poll的区别

  • 相比于selectpollepoll最大的好处载于它不会随着监听套接字fd数目的增多而降低效率。

  • 内核中的selectpoll的实现是采用轮询来处理的,轮询的fd数目越多,自然消耗时间越长

  • epoll的实现是基于回调的,如果fd有期望的事件发生就通过回调函数将其加入epoll就绪队列中,也就是说它只关心活跃的fd,与fd的数目无关。

  • 内核/用户空间 内存拷贝问题,如何让内核把fd消息通知给用户空间呢?

    在这个问题上select/poll采用了内存拷贝方法,而epoll采用了共享内存的方法。

  • epoll不仅会告诉应用程序有I/O事件,还会告诉应用程序相关的信息,这些信息是应用程序填充的,因此根据这些信息应用程序就能直接定位到事件,而不必遍历整个fd集合了。

学后感!!!

个人感觉epoll的机制和Qt的信号和槽的处理机制很有共同点,这里说的是思想方面,不是底层代码的实现和一些细节性的问题,比如说Qt的信号和槽的设计思想,当有信号时,才会去调用槽(槽就是一回调函数的,入口指针),节省了不停的轮巡遍历的操作,大大节省了CPU和内存的资源,因为你活跃起来了我再去处理你,否则的话,你就老老实实的待在哪里等信号。个人观点如有理解错误,请多多包含,同时希望可以指正。

##epoll LT/ET模式

  • EPOLLLT:电平触发模式
  • EPOLLET:边沿触发模式,效率高。

EPOLLLT模式

电平/水平触发模式,完全靠kernel epoll驱动,应用程序只需要处理从epoll_wait返回的fds,这些fds我们认为它们处于就绪状态

EPOLLET模式

边沿触发模式,是一把双刃剑,如果处理不当可能会导致效率不如,LT模式。如果需要高效率,则程序员则需要更高的要求,来维护ET模式

​ 此模式下,系统仅通知应用程序那些fds变成了就绪状态,一旦fd变成就绪状态,epoll将不再关注这个fd的任何状态信息(从epoll队列中移除),直到应用程序通过读写操作触发EAGAIN状态,epoll认为这个fd又变为空闲状态,那么epoll又重新关注这个fd的状态变话(添加到epoll队列中)

​ 随着epoll_wait的返回,队列中的fds是在减少的,所以在大并发的系统中,EPOLLET模式更具有优势,但是对程序员的要求也变高了。(emmm, 这是什么鬼啊),程序员需要对程序维护,避免读取一部分数据后剩下数据无法读出导致阻塞。

16 UNIX域协议学习

day16

本章目标

  • UNIX域协议特点
  • UNIX域地址结构
  • UNIX域字节流回射客户/服务
  • UNIX域套接字编程注意点

UNIX域协议特点

  • UNIX域套接字与TCP套接字相比较,在同一台主机的传输速度前者是后者的两倍。
  • UNIX域套接字可以在同一台主机上各进程之间传递描述符。
  • UNIX域套接字与传统套接字的区别是用路径名来表示协议族的描述。

UNIX域地址结构

man unix

1
2
3
4
5
6
#define UNIX_PATH_MAX 108
struct sockaddr_un
{
sa_family_t sun_family; //为AF_UNIX
char sun_path[UNIX_PATH_MAX]; //pathname
}

##UNIX域字节流回射客户/服务

echosrv.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <string.h>
#include <sys/un.h>

#define ERR_EXIT(x) do{perror(x); exit(EXIT_FAILURE);}while(0)

void echo_srv(int conn)
{
int ret = 0;
char recvbuf[1024] = {0};
while(1)
{
memset(recvbuf, 0, sizeof(recvbuf));
ret = read(conn, recvbuf, sizeof(recvbuf));
if(ret < 0)
{
if(errno = EINTR)
continue;
ERR_EXIT("read");
}
else if(ret == 0)
{
printf("对方关闭连接");
break;
}
fputs(recvbuf, stdout);
ret = write(conn, recvbuf, strlen(recvbuf));
}
close(conn);
}


int main()
{
int conn;
int listenfd;
unlink("test_socket");//删除一个文件的目录项并减少它的链接数,若成功则返回0,否则返回-1,错误原因存于error。如果想通过调用这个函数来成功删除文件,你就必须拥有这个文件的所属目录的写和执行权
if((listenfd = socket(PF_UNIX, SOCK_STREAM, 0)) < 0)
ERR_EXIT("socket");
struct sockaddr_un servaddr;
memset(&servaddr, 0, sizeof(servaddr));
servaddr.sun_family = AF_UNIX;
strcpy(servaddr.sun_path, "test_socket");
if(bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0)
ERR_EXIT("bind");
if(listen(listenfd, SOMAXCONN) < 0)
ERR_EXIT("listen");

pid_t pid;
while(1)
{
conn = accept(listenfd, NULL, NULL);
if(conn == -1)
{
//用户态资源并没有准备好,返回EINTR错误。以便用户态可以做出自己的决定。
if(errno = EINTR)
continue;
ERR_EXIT("accept");
}
pid = fork();
if(pid < 0)
ERR_EXIT("fork");
else if(pid == 0)//子进程处理新的连接
{
close(listenfd);//子进程不需要处理监听
echo_srv(conn);
exit(EXIT_SUCCESS);
}
close(conn);//父进程不需要处理连接
}
return 0;
}

echocli.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <string.h>
#include <sys/un.h>

#define ERR_EXIT(x) do{perror(x); exit(EXIT_FAILURE);}while(0)

void echo_cli(int sock)
{
int ret = 0;
char sendbuf[1024] = {0};
char recvbuf[1024] = {0};
while(fgets(sendbuf, sizeof(sendbuf), stdin) != NULL)
{
ret = write(sock, sendbuf, sizeof(sendbuf));
ret = read(sock, recvbuf, sizeof(recvbuf));
fputs(recvbuf, stdout);
memset(sendbuf, 0, sizeof(sendbuf));
memset(recvbuf, 0, sizeof(recvbuf));
}
close(sock);
}
int main()
{
int sock;
if((sock = socket(PF_UNIX, SOCK_STREAM, 0)) < 0)
ERR_EXIT("socket");
struct sockaddr_un servaddr;
memset(&servaddr, 0, sizeof(servaddr));
servaddr.sun_family = AF_UNIX;
strcpy(servaddr.sun_path, "test_socket");
if(connect(sock, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0)
ERR_EXIT("connect");

echo_cli(sock);
return 0;
}

##UNIX套接字编程注意点

  • bind成功将会创建一个文件,权限为0777&umask
  • sun_path最好用一个绝对路径,一般定到tmp目录下。
  • UNIX域协议支持流式接口SOCK_STREAM与报式套接口SOCK_DGRAM
  • UNIX域流式套接字connect发现监听队列满时,会立即返回一个ECONNERFUSED,这和TCP不同,如果监听套接字队列满,会忽略到来的SYN,导致对方重发SYN

17 socketpair、全双工流管道

day17

本章目标

  • socketpair:全双工的流管道
  • sendmsgrecvmsg
  • UNIX域套接字传递描述符字

socketpair

  • 功能:创建一个全双工的流管道,只能用于亲缘进程通信。
  • 原型
    • int socketpair(int domain, int type, int protocol, intsv[2]);
  • 参数
    • domain:协议家族
    • type:套接字类型
    • protocol:协议类型
    • sv:返回套接字对,两个套接字可读可写。
  • 返回值:成功返回0,失败返回-1

sendmsg和recvmsg

sendmsgssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
recvmsg ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);

主要为了用UNIX域协议传递套接字,sockfd只能套接字,不能是文件描述符。

struct iovec *msg_iov; /* scatter/gather array */与readv和writev有关,真正所发送的数据。

由于存在字节对齐,所以可能会有填充字节。

1
2
3
4
5
6
7
8
9
struct msghdr {
void *msg_name; /* optional address */
socklen_t msg_namelen; /* size of address */
struct iovec *msg_iov; /* scatter/gather array */与readv和writev有关,真正所发送的数据。struct iovec
size_t msg_iovlen; /* # elements in msg_iov */发送的iovec()的个数
void *msg_control; /* ancillary data, see below */辅助控制信息,为struct cmsghdr
size_t msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};

struct msghdr的成员变量结构体。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// struct iovec *msg_iov;  
struct iovec { /* Scatter/gather array items */
void *iov_base; /* Starting address */
size_t iov_len; /* Number of bytes to transfer */
};
//void *msg_control;
struct cmsghdr {
size_t cmsg_len; /* Data byte count, including header (type is socklen_t in POSIX) */
int cmsg_level; /* Originating protocol */
int cmsg_type; /* Protocol-specific type */
/* followed by
unsigned char cmsg_data[];
*/
};

readvwritevstruct iovec有关。

1
2
ssize_t readv(int fd, const struct iovec *iov, int iovcnt);
ssize_t writev(int fd, const struct iovec *iov, int iovcnt);

man CMSG_FIRSTHDR

参考文档:

https://blog.csdn.net/sparkliang/article/details/5486069

非亲缘进程之间传递套接字。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#include <sys/socket.h>

struct cmsghdr *CMSG_FIRSTHDR(struct msghdr *msgh);
struct cmsghdr *CMSG_NXTHDR(struct msghdr *msgh, struct cmsghdr *cmsg);
size_t CMSG_ALIGN(size_t length);
size_t CMSG_SPACE(size_t length);
size_t CMSG_LEN(size_t length);
unsigned char *CMSG_DATA(struct cmsghdr *cmsg);

struct cmsghdr {
size_t cmsg_len; /* Data byte count, including header
(type is socklen_t in POSIX) */
int cmsg_level; /* Originating protocol */
int cmsg_type; /* Protocol-specific type */
/* followed by
unsigned char cmsg_data[]; */
};

插播一个vim

vim的插入模式下按insert键,可以进入replace模式

参考文档:https://blog.csdn.net/overstack/article/details/9174693

多行复制:

1
2
3
例如要将第5到10行黏贴到第15行后面可以这么写:5,10 copy 15 这个方法适合有大量的行数情况。
move:5,10 move 15是移动,类似剪切+复制
delete为删除:5,10 delete删除5到10行

sendmsg和recvmsg应用代码

sendmsgrecvmsg是在父子进程间传递文件描述符的,不能只通过传递文件描述符的数值来传递给别的进程,因为你虽然传递了‘文件描述符’,但是这个只是数值上和文件描述符相等,并不代表真的传递过去了一个文件描述符,举个简单的例子:大学都需要考英语四级证书,如果说有人考过了四级,你没有去参加考试,你去把那个人的四级证书复制了一份改成你的名字了,那么不代表你也通过了大学英语四级考试,你只是有一个英语四级证书,而在存储英语四级证书通过人的数据库里面没有你的信息存在。所以你并没有通过四级考试,同理,进程间传递文件描述符,也不能只是简简单单的传递一个fd(这里指那个fd数字),在你看来fd代表了这个文件,其实不然,在这个fd的背后有着系统对这个fd的一些系列服务的内容,比如你打开了一个文件那么系统就为了维护了这个文件的相关信息,而数字fd系统不过是提供了一个友好的接口来让你方便操作你的那个文件。

且不能通过TCPUCP传递文件描述符。非亲缘进程的文件描述符的传递只能通过UNIX域协议来传递。

父子进程可以通过socketpair来传递,下面的代码是父子进程通过socketpair来传递文件描述符的。

send_recv_msg.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <string.h>
#include <sys/un.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>


#define ERR_EXIT(x) do{perror(x); exit(EXIT_FAILURE);}while(0)


void send_fd(int sock_fd, int send_fd)
{
int ret;
struct msghdr msg;
struct cmsghdr *p_cmsg;//struct msghdr的第五个成员变量msg_control的辅助控制信息,p_cmsg需要指向一块缓冲区
struct iovec vec;
char cmsgbuf[CMSG_SPACE(sizeof(send_fd))];//使用send_fd通过CMSG_SPACE算出来辅助空间的大小,CMEG_SPACE是一个宏.
int * p_fds;
char sendchar = 0;

msg.msg_control = cmsgbuf;//将msg_control指向cmsgbuf辅助数据
msg.msg_controllen = sizeof(cmsgbuf);//长度为sizeof(cmsgbuf)

p_cmsg = CMSG_FIRSTHDR(&msg);//通过宏CMSG_FIRSTHDR获取msg里面的第一个消息,通过CMSGNXTHDR获取第二个消息,特此说明:是CMSGNXTHDR不是CMSGNEXTHDR这个宏的NEXT没有E。
p_cmsg->cmsg_level = SOL_SOCKET;//和下面一句共同表示传递的是文件描述字
p_cmsg->cmsg_type = SCM_RIGHTS;
p_cmsg->cmsg_len = CMSG_LEN(sizeof(send_fd));//通过宏得到长度
p_fds = (int *)CMSG_DATA(p_cmsg);//得到数据的首地址
*p_fds = send_fd;//真正的存放数据的步骤。

//填充struct msghdr结构体变量msg
msg.msg_name = NULL;
msg.msg_namelen = 0;
msg.msg_iov = &vec;
msg.msg_iovlen = 1;//发送一个字符所以为1
msg.msg_flags = 0;

//填充struct iovec结构体变量vec
vec.iov_base = &sendchar;//因为只发送一个字符
vec.iov_len = sizeof(sendchar);//大小为一个字节

//数据终于在这里填充完毕了,可以发送数据了
ret = sendmsg(sock_fd, &msg, 0);
if(ret != 1)
{
ERR_EXIT("sendmsg");
}
}

int recv_fd(int sock_fd)
{
int ret;
struct msghdr msg;
char recvchar;
struct iovec vec;
int recv_fd;
char cmsgbuf[CMSG_SPACE(sizeof(send_fd))];//使用send_fd通过CMSG_SPACE算出来辅助空间的大小,CMEG_SPACE是一个宏.
struct cmsghdr *p_cmsg;//struct msghdr的第五个成员变量msg_control的辅助控制信息,p_cmsg需要指向一块缓冲区
int * p_fds;

//填充struct iovec结构体变量vec
vec.iov_base = &recvchar;//因为只发送一个字符
vec.iov_len = sizeof(recvchar);//大小为一个字节

//填充struct msghdr结构体变量msg
msg.msg_name = NULL;
msg.msg_namelen = 0;
msg.msg_iov = &vec;
msg.msg_iovlen = 1;//发送一个字符所以为1
msg.msg_flags = 0;
msg.msg_control = cmsgbuf;//将msg_control指向cmsgbuf辅助数据
msg.msg_controllen = sizeof(cmsgbuf);//长度为sizeof(cmsgbuf)
msg.msg_flags = 0;

p_fds = (int *)(CMSG_DATA(CMSG_FIRSTHDR(&msg)));//得到数据的首地址
*p_fds = -1;//真正的存放数据的步骤。

//数据终于在这里填充完毕了,可以接收数据了
ret = recvmsg(sock_fd, &msg, 0);
if(ret != 1)
{
ERR_EXIT("recvmsg");
}
p_cmsg = CMSG_FIRSTHDR(&msg);
if(p_cmsg == NULL)
{
-ERR_EXIT("CMSG_FIRSTHDR");
}

p_fds = (int *)CMSG_DATA(p_cmsg);
recv_fd = *p_fds;
if(recv_fd == -1)
{
ERR_EXIT("CMSG_DATA");
}
return recv_fd;
}

测试代码

send_recv_msg.c

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <string.h>
#include <sys/un.h>
#include "sendmsg.h"
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>


#define ERR_EXIT(x) do{perror(x); exit(EXIT_FAILURE);}while(0)

void send_fd(int sock_fd, int send_fd);
int recv_fd(int sock_fd);

int main()
{
int sockfds[2];
if(socketpair(PF_UNIX, SOCK_STREAM, 0, sockfds) < 0)
ERR_EXIT("sockfdsetpair");
pid_t pid;
pid = fork();
if(pid < 0)
{
ERR_EXIT("fork");
}
else if(pid == 0)
{
//close(sockfds[1]);
int fd;
fd = open("text.txt", O_RDONLY);
if(fd == -1)
{
ERR_EXIT("open");
}
send_fd(sockfds[0], fd);
}
else
{
char buf[1024];
//close(sockfds[0]);
int fd = recv_fd(sockfds[1]);
read(fd, buf, sizeof(buf));
printf("buf = %s\n", buf);
}
return 0;
}

01 Golang初识

01 Go初识

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
package main

import "fmt"

/*
* 三种不同的打印语句
* 1. fmt.Println(),可以打印字符串和变量,输出后自动增加一个换行
* 2. fmt.Printf(),格式化输出
* 3. fmt.Print(),打印字符串常量
**/

func main() { // { 不能单独放一行
var i int
for i = 1; i <= 9; i++ {
for j := 1; j <= i ; j++ {
// fmt.Print(i, "*", j, "=", i*j, " ")
fmt.Printf("%d * %d =%2d ", i, j, i*j)
// fmt.Println(i, "*", j, "=", i*j, " ") // 最后会换行,无法格式打印9x9乘法表
}
fmt.Println()
}
}