那时我以为我是风,可以自由的飘向远方……

技术文摘

如何在Linux下获得一些中国电信运营商的IP地址分配情况

APNIC是管理亚太地区IP地址分配的机构,它有着丰富准确的IP地址分配库,同时这些信息也是对外公开的,并提供了一个查询工具,下面就让我们看看如何在Linux下获得一些中国基础电信运营商(网通、电信、铁通、教育网)的IP地址分配情况:

wget http://ftp.apnic.net/apnic/dbase/tools/ripe-dbase-client-v3.tar.gz
tar xzvf ripe-dbase-client-v3.tar.gz
cd whois-3.1
./configure
make
make install
编译安装后,我们可以通过以下命令来获取某个运营商的IP地址段;

中国网通:
./whois3 -h whois.apnic.net -l -imb MAINT-CNCGROUP>/var/cnc

中国电信:
./whois3 -h whois.apnic.net -l -imb MAINT-CHINANET>/var/chinanet

中国铁通:
./whois3 -h whois.apnic.net -l -imb MAINT-CN-CRTC>/var/crtc
打开获取后的文件可以看到里面的信息非常详细,甚至可以看到各个分公司的负责人、电话、电子邮件等等信息。如果想得到一份整齐干净的IP地址段文件,只要用grep和awk简单过滤就可以了
如果想得到具体的服务商比如江苏省电信的IP池,就把mb的值改为MAINT-CHINANET-JS,或者是辽宁网通,那就改为MAINT-CNCGROUP-LN

Active FTP vs. Passive FTP, a Definitive Explanation

Contents:

 
 

Introduction

One of the most commonly seen questions when dealing with firewalls and other Internet connectivity issues is the difference between active and passive FTP and how best to support either or both of them. Hopefully the following text will help to clear up some of the confusion over how to support FTP in a firewalled environment.

This may not be the definitive explanation, as the title claims, however, I’ve heard enough good feedback and seen this document linked in enough places to know that quite a few people have found it to be useful. I am always looking for ways to improve things though, and if you find something that is not quite clear or needs more explanation, please let me know! Recent additions to this document include the examples of both active and passive command line FTP sessions. These session examples should help make things a bit clearer. They also provide a nice picture into what goes on behind the scenes during an FTP session. Now, on to the information…
 

The Basics

FTP is a TCP based service exclusively. There is no UDP component to FTP. FTP is an unusual service in that it utilizes two ports, a ‘data’ port and a ‘command’ port (also known as the control port). Traditionally these are port 21 for the command port and port 20 for the data port. The confusion begins however, when we find that depending on the mode, the data port is not always on port 20.
 

Active FTP

In active mode FTP the client connects from a random unprivileged port (N > 1023) to the FTP server’s command port, port 21. Then, the client starts listening to port N+1 and sends the FTP command PORT N+1 to the FTP server. The server will then connect back to the client’s specified data port from its local data port, which is port 20.

From the server-side firewall’s standpoint, to support active mode FTP the following communication channels need to be opened:

  • FTP server’s port 21 from anywhere (Client initiates connection)
  • FTP server’s port 21 to ports > 1023 (Server responds to client’s control port)
  • FTP server’s port 20 to ports > 1023 (Server initiates data connection to client’s data port)
  • FTP server’s port 20 from ports > 1023 (Client sends ACKs to server’s data port)

 

When drawn out, the connection appears as follows:

In step 1, the client’s command port contacts the server’s command port and sends the command PORT 1027. The server then sends an ACK back to the client’s command port in step 2. In step 3 the server initiates a connection on its local data port to the data port the client specified earlier. Finally, the client sends an ACK back as shown in step 4.

 

The main problem with active mode FTP actually falls on the client side. The FTP client doesn’t make the actual connection to the data port of the server–it simply tells the server what port it is listening on and the server connects back to the specified port on the client. From the client side firewall this appears to be an outside system initiating a connection to an internal client–something that is usually blocked.
 

Active FTP Example

Below is an actual example of an active FTP session. The only things that have been changed are the server names, IP addresses, and user names. In this example an FTP session is initiated from testbox1.slacksite.com (192.168.150.80), a linux box running the standard FTP command line client, to testbox2.slacksite.com (192.168.150.90), a linux box running ProFTPd 1.2.2RC2. The debugging (-d) flag is used with the FTP client to show what is going on behind the scenes. Everything in red is the debugging output which shows the actual FTP commands being sent to the server and the responses generated from those commands. Normal server output is shown in black, and user input is in bold.

There are a few interesting things to consider about this dialog. Notice that when the PORT command is issued, it specifies a port on the client (192.168.150.80) system, rather than the server. We will see the opposite behavior when we use passive FTP. While we are on the subject, a quick note about the format of the PORT command. As you can see in the example below it is formatted as a series of six numbers separated by commas. The first four octets are the IP address while the last two octets comprise the port that will be used for the data connection. To find the actual port multiply the fifth octet by 256 and then add the sixth octet to the total. Thus in the example below the port number is ( (14*256) + 178), or 3762. A quick check with netstat should confirm this information.

testbox1: {/home/p-t/slacker/public_html} % ftp -d testbox2
Connected to testbox2.slacksite.com.
220 testbox2.slacksite.com FTP server ready.
Name (testbox2:slacker): slacker
—> USER slacker
331 Password required for slacker.
Password: TmpPass
—> PASS XXXX
230 User slacker logged in.
—> SYST
215 UNIX Type: L8

Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
ftp: setsockopt (ignored): Permission denied
—> PORT 192,168,150,80,14,178

200 PORT command successful.
—> LIST
150 Opening ASCII mode data connection for file list.
drwx—— 3 slacker users 104 Jul 27 01:45 public_html
226 Transfer complete.
ftp> quit
—> QUIT
221 Goodbye.

 

Passive FTP

In order to resolve the issue of the server initiating the connection to the client a different method for FTP connections was developed. This was known as passive mode, or PASV, after the command used by the client to tell the server it is in passive mode.

In passive mode FTP the client initiates both connections to the server, solving the problem of firewalls filtering the incoming data port connection to the client from the server. When opening an FTP connection, the client opens two random unprivileged ports locally (N > 1023 and N+1). The first port contacts the server on port 21, but instead of then issuing a PORT command and allowing the server to connect back to its data port, the client will issue the PASV command. The result of this is that the server then opens a random unprivileged port (P > 1023) and sends the PORT P command back to the client. The client then initiates the connection from port N+1 to port P on the server to transfer data.

From the server-side firewall’s standpoint, to support passive mode FTP the following communication channels need to be opened:

  • FTP server’s port 21 from anywhere (Client initiates connection)
  • FTP server’s port 21 to ports > 1023 (Server responds to client’s control port)
  • FTP server’s ports > 1023 from anywhere (Client initiates data connection to random port specified by server)
  • FTP server’s ports > 1023 to remote ports > 1023 (Server sends ACKs (and data) to client’s data port)

 

When drawn, a passive mode FTP connection looks like this:

In step 1, the client contacts the server on the command port and issues the PASV command. The server then replies in step 2 with PORT 2024, telling the client which port it is listening to for the data connection. In step 3 the client then initiates the data connection from its data port to the specified server data port. Finally, the server sends back an ACK in step 4 to the client’s data port.

 

While passive mode FTP solves many of the problems from the client side, it opens up a whole range of problems on the server side. The biggest issue is the need to allow any remote connection to high numbered ports on the server. Fortunately, many FTP daemons, including the popular WU-FTPD allow the administrator to specify a range of ports which the FTP server will use. See Appendix 1 for more information.

The second issue involves supporting and troubleshooting clients which do (or do not) support passive mode. As an example, the command line FTP utility provided with Solaris does not support passive mode, necessitating a third-party FTP client, such as ncftp.

With the massive popularity of the World Wide Web, many people prefer to use their web browser as an FTP client. Most browsers only support passive mode when accessing ftp:// URLs. This can either be good or bad depending on what the servers and firewalls are configured to support.
 

Passive FTP Example

Below is an actual example of a passive FTP session. The only things that have been changed are the server names, IP addresses, and user names. In this example an FTP session is initiated from testbox1.slacksite.com (192.168.150.80), a linux box running the standard FTP command line client, to testbox2.slacksite.com (192.168.150.90), a linux box running ProFTPd 1.2.2RC2. The debugging (-d) flag is used with the FTP client to show what is going on behind the scenes. Everything in red is the debugging output which shows the actual FTP commands being sent to the server and the responses generated from those commands. Normal server output is shown in black, and user input is in bold.

Notice the difference in the PORT command in this example as opposed to the active FTP example. Here, we see a port being opened on the server (192.168.150.90) system, rather than the client. See the discussion about the format of the PORT command above, in the Active FTP Example section.

testbox1: {/home/p-t/slacker/public_html} % ftp -d testbox2
Connected to testbox2.slacksite.com.
220 testbox2.slacksite.com FTP server ready.
Name (testbox2:slacker): slacker
—> USER slacker
331 Password required for slacker.
Password: TmpPass
—> PASS XXXX
230 User slacker logged in.
—> SYST
215 UNIX Type: L8

Remote system type is UNIX.
Using binary mode to transfer files.
ftp> passive
Passive mode on.
ftp> ls
ftp: setsockopt (ignored): Permission denied
—> PASV

227 Entering Passive Mode (192,168,150,90,195,149).
—> LIST
150 Opening ASCII mode data connection for file list
drwx—— 3 slacker users 104 Jul 27 01:45 public_html
226 Transfer complete.
ftp> quit —> QUIT
221 Goodbye.

 

Other Notes

A reader, Maarten Sjouw, pointed out that active FTP will not function when used in conjunction with a client-side NAT (Network Address Translation) device which is not smart enough to alter the IP address info in FTP packets.
 

Summary

The following chart should help admins remember how each FTP mode works:

Active FTP :
command : client >1023 -> server 21
data : client >1023 < - server 20

Passive FTP :
command : client >1023 -> server 21
data : client >1023 -> server >1023

 

A quick summary of the pros and cons of active vs. passive FTP is also in order:

Active FTP is beneficial to the FTP server admin, but detrimental to the client side admin. The FTP server attempts to make connections to random high ports on the client, which would almost certainly be blocked by a firewall on the client side. Passive FTP is beneficial to the client, but detrimental to the FTP server admin. The client will make both connections to the server, but one of them will be to a random high port, which would almost certainly be blocked by a firewall on the server side.

Luckily, there is somewhat of a compromise. Since admins running FTP servers will need to make their servers accessible to the greatest number of clients, they will almost certainly need to support passive FTP. The exposure of high level ports on the server can be minimized by specifying a limited port range for the FTP server to use. Thus, everything except for this range of ports can be firewalled on the server side. While this doesn’t eliminate all risk to the server, it decreases it tremendously. See Appendix 1 for more information.

References

An excellent reference on how various internet protocols work and the issues involved in firewalling them can be found in the O’Reilly and Associates book, Building Internet Firewalls, 2nd Ed, by Brent Chapman and Elizabeth Zwicky.

Finally, the definitive reference on FTP would be RFC 959, which sets forth the official specifications of the FTP protocol. RFCs can be downloaded from numerous locations, including http://www.faqs.org/rfcs/rfc959.html.

from:http://slacksite.com/other/ftp.html

vsftp的安装和配置

在因特网上或是企业内部,有许多站点需要高质量的FTP应用和安全的服务控制,如何能配置高质量应用的安全站点是企业应用和一些提供下载服务的网站的重要需求。基于这个出发点,我们将使用VSFTP–very safe ftp–架设高质量应用的安全FTP站点。
Read more »

从LiveJournal后台发展看大规模网站性能优化方法

一、LiveJournal发展历程

LiveJournal是99年始于校园中的项目,几个人出于爱好做了这样一个应用,以实现以下功能:

  • 博客,论坛
  • 社会性网络,找到朋友
  • 聚合,把朋友的文章聚合在一起

LiveJournal采用了大量的开源软件,甚至它本身也是一个开源软件。在上线后,LiveJournal实现了非常快速的增长:

  • 2004年4月份:280万注册用户。
  • 2005年4月份:680万注册用户。
  • 2005年8月份:790万注册用户。
  • 达到了每秒钟上千次的页面请求及处理。
  • 使用了大量MySQL服务器。
  • 使用了大量通用组件。 Read more »

使用memcached进行内存缓存

通常的网页缓存方式有动态缓存和静态缓存等几种,在ASP.NET中已经可以实现对页面局部进行缓存,而使用memcached的缓存比ASP.NET的局部缓存更加灵活,可以缓存任意的对象,不管是否在页面上输出。而memcached最大的优点是可以分布式的部署,这对于大规模应用来说也是必不可少的要求。
LiveJournal.com使用了memcached在前端进行缓存,取得了良好的效果,而像wikipedia,sourceforge等也采用了或即将采用memcached作为缓存工具。memcached可以大规模网站应用发挥巨大的作用。 Read more »

ASP开发中存储过程应用全接触

考研淘吧在这次的升级中大量应用到存储过程,为了避免以后的遗忘,转载一篇有关存储过程的文章,留待随时查阅。

ASP与存储过程(Stored Procedures)的文章不少,但是我怀疑作者们是否真正实践过。我在初学时查阅过大量相关资料,发现其中提供的很多方法实际操作起来并不是那么回事。对于简单的应用,这些资料也许是有帮助的,但仅限于此,因为它们根本就是千篇一律,互相抄袭,稍微复杂点的应用,就全都语焉不详了。

  现在,我基本上通过调用存储过程访问SQL Server,以下的文字都是实践的总结,希望对大家能有帮助。

  存储过程就是作为可执行对象存放在数据库中的一个或多个SQL命令。

  定义总是很抽象。存储过程其实就是能完成一定操作的一组SQL语句,只不过这组语句是放在数据库中的(这里我们只谈SQL Server)。如果我们通过创建存储过程以及在ASP中调用存储过程,就可以避免将SQL语句同ASP代码混杂在一起。这样做的好处至少有三个:

  第一、大大提高效率。存储过程本身的执行速度非常快,而且,调用存储过程可以大大减少同数据库的交互次数。

  第二、提高安全性。假如将SQL语句混合在ASP代码中,一旦代码失密,同时也就意味着库结构失密。

  第三、有利于SQL语句的重用。
Read more »

msxml3.dll 错误 ‘800c0005’解决方案

在运用xmlhttp组件编写程序中,会碰到 ”msxml3.dll 错误 ‘800c0005’ 系统未找到指定的资源。” 这种错误,网上对这种错误的产生原因有很多钟解释,大体说是因为防火墙或UDP站口权限造成了,也说了相应的解决办法。其它有时候也未必。其实错误的描述中就说出了主要的原因 ”系统未找到指定的资源” 。这种错误都是出现在调用了 xmlhttp 组件的 Open方法,接着再用Send方法后造成的。当open方法的的 url 参数无法访问时,就会造成 8000005 错误。并且一旦产生这种错误,就会导致应用程序终止,无法继续操作。 Read more »

可缓存的动态页面设计

什么样的页面能够比较好的被缓存服务器缓存呢?如果返回内容的HTTP HEADER中有”Last-Modified”和”Expires”相关声明,比如:

Last-Modified: Wed, 14 May 2003 13:06:17 GMT
Expires: Fri, 16 Jun 2003 13:06:17 GMT

前端缓存服务器在期间会将生成的页面缓存在本地:硬盘或者内存中,直至上述页面过期。 Read more »

面向缓存的站点规划

一个利用SQUID对多个站点进行做WEB加速http acceleration方案:
原先一个站点的规划可能是这样的:
200.200.200.207 www.chedong.com
200.200.200.208 news.chedong.com
200.200.200.209 bbs.chedong.com
200.200.200.205 images.chedong.com
面向缓存服务器的设计中:所有站点都通过外部DNS指向到同一个IP:200.200.200.200/201这2台缓存服务器上(使用2台是为了冗余备份) Read more »

基于Squid的反向代理加速实现

Squid是一个更专用的代理服务器,性能和效率会比Apache的mod_proxy高很多。

Read more »