[Note figure for estimated total world news storage.] From: reid@pa.dec.com (Brian Reid) Newsgroups: news.groups,news.lists,news.admin.misc,news.lists.ps-maps Subject: USENET FLOW ANALYSIS for JAN 94: Who stores how much news Date: 12 Feb 1994 20:49:42 -0800 Organization: DEC Network Systems Laboratory Analysis of stored news articles, JAN 94. This is an analysis of the contents of /usr/spool/news at the sites reporting "inpaths" data. The "inpaths" program has been posted in news.lists.ps-maps,comp.sources.d,news.admin.misc. Please consider installing and running this program at your site. Presumed size of overall network: 140000 Number of sites surveyed: 505 (0.4%) Average age of articles kept online: 8.1 days Average age of disk space used by news: 10.7 days Average disk space used by news: 282.3 megabytes Average number of articles stored: 94087 Estimated worldwide disk space used by news 40 terabytes Distribution of expiration times used 0-1 10 ************ 1-2 13 **************** 2-3 25 ******************************* 3-4 22 **************************** 4-5 32 **************************************** 5-6 35 ******************************************** 6-7 38 ************************************************ 7-8 39 ************************************************* 8-9 32 **************************************** 9-10 47 ************************************************************ 10-12 36 ********************************************* 12-14 29 ************************************* 14-16 29 ************************************* 16-18 18 ********************** 18-20 30 ************************************** 20-25 15 ******************* 25-30 12 *************** 30-35 12 *************** 35-40 5 ****** 40-45 3 *** 45-50 11 ************** 50-75 1 * 75-100 1 * 100-125 0 125-150 1 * 150-175 3 *** 175-200 2 ** 200-225 0 225-250 0 250-275 0 275-300 1 * Notes: The "average disk space used by news" assumes that the host operating system allocates disk space in a fixed "chunk size" of about 1000 bytes. The size of each stored message is rounded up to the next multiple of that chunk size. The "average age of articles" is an average counting each article equally. The "average age of disk space" is weighted by size. The "expiration time" for a site is not necessarily constant. Some newsgroups that are considered more valuable are given longer expiration times, while obvious junk is given a shorter expiration time. To come up with a single expiration time for a site, we find the average age of stored articles and then double it, rounding to the nearest integer. If you expire all articles older than 14 days, and if the arrival rate is constant, then the average age of articles at your site will be 7 days.