Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh
 |
文章出处: |
{{#if: | {{{2}}} | http://wiki.ubuntu.org.cn/Quick_HOWTO_:_Ch05_:_Troubleshooting_Linux_with_syslog }} |
|
翻译人员: |
lzhonsailing |
|
点击翻译: |
English {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/af | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Afrikaans| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/af|Afrikaans]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ar | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|العربية| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ar|العربية]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/az | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|azərbaycanca| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/az|azərbaycanca]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/bcc | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|جهلسری بلوچی| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/bcc|جهلسری بلوچی]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/bg | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|български| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/bg|български]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/br | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|brezhoneg| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/br|brezhoneg]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ca | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|català| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ca|català]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/cs | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|čeština| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/cs|čeština]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/de | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Deutsch| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/de|Deutsch]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/el | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Ελληνικά| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/el|Ελληνικά]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/es | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|español| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/es|español]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/fa | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|فارسی| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/fa|فارسی]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/fi | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|suomi| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/fi|suomi]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/fr | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|français| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/fr|français]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/gu | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|ગુજરાતી| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/gu|ગુજરાતી]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/he | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|עברית| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/he|עברית]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/hu | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|magyar| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/hu|magyar]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/id | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Bahasa Indonesia| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/id|Bahasa Indonesia]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/it | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|italiano| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/it|italiano]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ja | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|日本語| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ja|日本語]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ko | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|한국어| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ko|한국어]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ksh | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Ripoarisch| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ksh|Ripoarisch]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/mr | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|मराठी| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/mr|मराठी]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ms | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Bahasa Melayu| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ms|Bahasa Melayu]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/nl | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Nederlands| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/nl|Nederlands]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/no | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|norsk| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/no|norsk]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/oc | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|occitan| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/oc|occitan]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/pl | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|polski| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/pl|polski]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/pt | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|português| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/pt|português]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ro | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|română| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ro|română]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/ru | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|русский| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/ru|русский]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/si | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|සිංහල| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/si|සිංහල]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/sq | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|shqip| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/sq|shqip]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/sr | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|српски / srpski| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/sr|српски / srpski]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/sv | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|svenska| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/sv|svenska]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/th | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|ไทย| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/th|ไทย]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/tr | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Türkçe| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/tr|Türkçe]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/vi | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|Tiếng Việt| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/vi|Tiếng Việt]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/yue | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|粵語| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/yue|粵語]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/zh | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|中文| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/zh|中文]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/zh-hans | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|中文（简体）| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/zh-hans|中文（简体）]]}}|}} {{#ifexist: {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | Quick HOWTO : Ch05 : Troubleshooting Linux with syslog | {{#if: | :}}Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh}}/zh-hant | • {{#if: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|中文（繁體）| [[::Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/zh-hant|中文（繁體）]]}}|}} |
{{#ifeq:Quick HOWTO : Ch05 : Troubleshooting Linux with syslog|:Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh|请不要直接编辑翻译本页，本页将定期与来源同步。}} |
{{#ifexist: :Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/zh | | {{#ifexist: Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh/zh | | {{#ifeq: {{#titleparts:Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh|1|-1|}} | zh | | }} }} }} {{#ifeq: {{#titleparts:Quick HOWTO : Ch05 : Troubleshooting Linux with syslog/zh|1|-1|}} | zh | | }}
简介
目前市场上有数百种 Linux 应用软件，它们都有自己的配置文件和帮助文件。虽然这种多样性显示了 Linux 领域的活力，但给系统管理造成了不少的麻烦。幸运的是，大部分 Linux 应用程序都使用 syslog 工具来记录它们运行时产生的所有错误和状态信息。这些记录文件被保存在 /var/log 目录中。
这些日志对于我们分析系统中发生的事件的时间和起因非常有价值。记住，尽管很多软件经常不把错误信息显示在屏幕上，但是它们还是会把这些日志记录下来。了解这些输出信息对于解决错误来说是非常重要的。因为你可以根据这些信息从诸如产品手册、在线文档以及网页搜索中迅速找到答案。
syslog，以及清除日志文件的工具 -- logrotate，它们配置起来相对比较容易，但它们分析起来还是有些难度。我接下来将用专门的一章来讲解 syslog。以此来强调它们对于你的 Linux 知识的重要性，并帮助你学会一项有用的技能来帮助你将来解决各种不同的软件错误。
syslog
syslog是一个追踪和记录系统运行时所产生的所有信息的工具。从简单报告到一重错误一概记录。每条系统信息在发送到syslog服务器时，都会加上两条相关的标签以便将来处理。
- 第一个标签描述的是产生这条信息的软件的功能。例如，一个邮件或者计划任务所产生的信息将分表表上mail和cron。
- 第二个标签描述的是这条信息的严重程度。一个有八个级别，列在表5-1中：
你可以修改syslog的配置文件来将不同严重程度和功能的信息分别储存到不同的文件中。这将会在以后详细说明。
表 5-1 Syslog Facilities
|
严重程度 |
关键字 |
描述 |
|
0 |
emergencies |
系统无法使用 |
|
1 |
alerts |
急需处理 |
|
2 |
critical |
危险 |
|
3 |
errors |
出错 |
|
4 |
warnings |
警告 |
|
5 |
notifications |
一般问题但是很重要 |
|
6 |
informational |
通知 |
|
7 |
debugging |
调试 |
/etc/syslog.conf 文件
在/etc/syslog.conf配置文件中，可以设置各类信息将写入哪个文件。这个配置文件包括两列。第一列列出了应用程序所属分类名和预期的严重程度，第二列是这些信息将记录到哪个文件。默认情况下，主RedHat/Fedora的/etc/syslog.conf文件设置为为把大部分信息记录在/var/log/messages文件中。这里是一个范例：
*.info;mail.none;authpriv.none;cron.none /var/log/messages
在这个例子中，除了功能为“邮件”、“权限确认”和“计划任务”产生的信息不记录外，其他所有的严重程度为“通知”的信息将会全部记录。你只要将上面这句进行如下修改，日志系统将会更加敏感。它将会记录所有严重程度等于或高于“调试”级别的信息。这样将会更有利于解决系统使用中出现的错误。
*.debug /var/log/messages
在下面这个例子中，除了auth, authpriv, news 和 mail等类别外，其他所有的”调试”级别的错误都将会记录到/var/log/debug文件中。值得注意的是，你可以用每一行结尾加一个反斜杠（\）的方法将一行内容写成多行。
*.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug
以下设置除auth,authpriv,news,mail类别外，其他所有类别的info、notice和warn级别的信息将会记录到/var/log/messages中。
*.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages
你也可以让某些特定的信息发送到所有登陆的用户屏幕上。以下这个例子可以将所有严重级别等于或高于“emergencies”的信息发送出去。你只需要用星号代替其他就可以了。
*.emerg *
某些应用软件也会将自己的运行状况记录到特定的日志文件上，而这个功能是不受以上介绍的syslog.conf文件控制的。举例如下：
文件:
/var/log/maillog : 邮件 /var/log/httpd/access_log : Apache网页服务器访问记录
文件夹:
/var/log /var/log/samba : Samba文件共享服务器信息 /var/log/mrtg : MRTG 信息 /var/log/httpd : Apache网页服务器信息
注意：在某些老版本的Linux中，/etc/syslog.conf文件只识别tab键间隔。如果用空格，将导致一些不可预料的后果。为了保证安全请详细检查/etc/syslog.conf文件的格式。
使syslog配置文件的修改生效
对/etc/syslog.conf文件的修改只有在你下次重启syslog服务时才会生效。使用以下命令可以实现：
[root@bigboy tmp]# service syslog restart
在 Ubuntu / Debian 系统中，请使用如下命令重启syslog服务：
root@u-bigboy:~# /etc/init.d/sysklogd restart
怎样查看日志文件里最新的记录
如果你想让最新的日志记录显示在屏幕上，可以用如下命令实现：
[root@bigboy tmp]# tail -f /var/log/messages
类似的命令可以用来查看所有日志文件。这有可能是Linux系统中最好的用来排除系统错误的工具。另外一个很好的工具是grep。grep可以帮你在一个日志文件中找到所有包含某个特定词条的事件；而且你还可以运用管道和more这个命令使得这些信息每次只显示一页：
[root@bigboy tmp]# grep string /var/log/messages | more
当然你也可以只简简单单的用more来将日志文件一次一屏的显示出来：
[root@bigboy tmp]# more /var/log/messages
将日志信息记录到远程Linux服务器上
Logging your system messages to a remote server is a good security practice. With all servers logging to a central syslog server, it becomes easier to correlate events across your company. It also makes covering up mistakes or malicious activities harder because the purposeful deletion of log files on a server cannot simultaneously occur on your logging server, especially if you restrict the user access to the logging server.
将你的系统日志记录到远程服务器上将是一个很好的安全措施。如果把所有
Configuring the Linux Syslog Server
By default syslog doesn't expect to receive messages from remote clients. Here's how to configure your Linux server to start listening for these messages.
As we saw previously, syslog checks its /etc/syslog.conf file to determine the expected names and locations of the log files it should create. It also checks the file /etc/sysconfig/syslog to determine the various modes in which it should operate. Syslog will not listen for remote messages unless the SYSLOGD_OPTIONS variable in this file has a -r included in it as shown below.
# Options to syslogd # -m 0 disables 'MARK' messages. # -r enables logging from remote machines # -x disables DNS lookups on messages received with -r # See syslogd(8) for more details SYSLOGD_OPTIONS="-m 0 -r" # Options to klogd # -2 prints all kernel oops messages twice; once for klogd to decode, and # once for processing with 'ksymoops' # -x disables all klogd processing of oops messages entirely # See klogd(8) for more details KLOGD_OPTIONS="-2"
Note: In Debian / Ubuntu systems you have to edit the syslog startup script /etc/init.d/sysklogd directly and make the SYSLOGD variable definition become "-r".
# Options for start/restart the daemons # For remote UDP logging use SYSLOGD="-r" # #SYSLOGD="-u syslog" SYSLOGD="-r"
You will have to restart syslog on the server for the changes to take effect. The server will now start to listen on UDP port 514, which you can verify using either one of the following netstat command variations.
[root@bigboy tmp]# netstat -a | grep syslog udp 0 0 *:syslog *:* [root@bigboy tmp]# netstat -an | grep 514 udp 0 0 0.0.0.0:514 0.0.0.0:* [root@bigboy tmp]#
Configuring the Linux Client
The syslog server is now expecting to receive syslog messages. You have to configure your remote Linux client to send messages to it. This is done by editing the /etc/hosts file on the Linux client named smallfry. Here are the steps:
1) Determine the IP address and fully qualified hostname of your remote logging host.
2) Add an entry in the /etc/hosts file in the format:
IP-address fully-qualified-domain-name hostname "loghost"
Example:
192.168.1.100 bigboy.my-site.com bigboy loghost
Now your /etc/hosts file has a nickname of "loghost" for server bigboy.
3) The next thing you need to do is edit your /etc/syslog.conf file to make the syslog messages get sent to your new loghost nickname.
*.debug @loghost *.debug /var/log/messages
You have now configured all debug messages and higher to be logged to both server bigboy ("loghost") and the local file /var/log/messages. Remember to restart syslog to get the remote logging started.
You can now test to make sure that the syslog server is receiving the messages with a simple test such as restarting the lpd printer daemon and making sure the remote server sees the messages.
Linux Client
[root@smallfry tmp]# service lpd restart Stopping lpd: [ OK ] Starting lpd: [ OK ] [root@smallfry tmp]#
Linux Server
[root@bigboy tmp]# tail /var/log/messages ... ... Apr 11 22:09:35 smallfry lpd: lpd shutdown succeeded Apr 11 22:09:39 smallfry lpd: lpd startup succeeded ... ... [root@bigboy tmp]#
Syslog Configuration and Cisco Network Devices
syslog reserves facilities "local0" through "local7" for log messages received from remote servers and network devices. Routers, switches, firewalls and load balancers each logging with a different facility can each have their own log files for easy troubleshooting. Appendix 4 has examples of how to configure syslog to do this with Cisco devices using separate log files for the routers, switches, PIX firewalls, CSS load balancers and LocalDirectors.
Logrotate
The Linux utility logrotate renames and reuses system error log files on a periodic basis so that they don't occupy excessive disk space.
The /etc/logrotate.conf File
This is logrotate's general configuration file in which you can specify the frequency with which the files are reused.
- You can specify either a weekly or daily rotation parameter. In the case below the weekly option is commented out with a #, allowing for daily updates.
- The rotate parameter specifies the number of copies of log files logrotate will maintain. In the case below the 4 copy option is commented out with a #, while allowing 7 copies.
- The create parameter creates a new log file after each rotation
Therefore, our sample configuration file will create daily archives of all the logfiles and store them for seven days. The files will have the following names with, logfile being current active version:
logfile logfile.0 logfile.1 logfile.2 logfile.3 logfile.4 logfile.5 logfile.6
Sample Contents of /etc/logrotate.conf
# rotate log files weekly #weekly # rotate log files daily daily # keep 4 weeks worth of backlogs #rotate 4 # keep 7 days worth of backlogs rotate 7 # create new (empty) log files after rotating old ones create
The /etc/logrotate.d Directory
Most Linux applications that use syslog will put an additional configuration file in this directory to specify the names of the log files to be rotated. It is a good practice to verify that all new applications that you want to use the syslog log have configuration files in this directory. Here are some sample files that define the specific files to be rotated for each application.
Here is an example of a custom file located in this directory that rotates files with the .tgz extension which are located in the /data/backups directory. The parameters in this file will override the global defaults in the /etc/logrotate.conf file. In this case, the rotated files won't be compressed, they'll be held for 30 days only if they are not empty, and they will be given file permissions of 600 for user root.
/data/backups/*.tgz {
daily
rotate 30
nocompress
missingok
notifempty
create 0600 root root
}
Note: In Debian / Ubuntu systems the /etc/cron.daily/sysklogd script reads the /etc/syslog.conf file and rotates any log files it finds configured there. This eliminates the need to create log rotation configuration files for the common system log files in the /etc/logrotate.d directory. As the script resides in the /etc/cron.daily directory it automatically runs every 24 hours. In Fedora / Redhat systems this script is replaced by the /etc/cron.daily/logrotate daily script which does not use the contents of the syslog configuration file, relying mostly on the contents of the /etc/logrotate.d directory.
Activating logrotate
The above logrotate settings in the previous section will not take effect until you issue the following command:
[root@bigboy tmp]# logrotate -f
If you want logrotate to reload only a specific configuration file, and not all of them, then issue the logrotate command with just that filename as the argument like this:
[root@bigboy tmp]# logrotate -f /etc/logrotate.d/syslog
Compressing Your Log Files
On busy Web sites the size of your log files can become quite large. Compression can be activated by editing the logrotate.conf file and adding the compress option.
# # File: /etc/logrotate.conf # # Activate log compression compress
The log files will then start to become archived with the gzip utility, each file having a .gz extension.
[root@bigboy tmp]# ls /var/log/messages* /var/log/messages /var/log/messages.1.gz /var/log/messages.2.gz /var/log/messages.3.gz /var/log/messages.4.gz /var/log/messages.5.gz /var/log/messages.6.gz /var/log/messages.7.gz [root@bigboy tmp]#
Viewing the contents of the files still remains easy because the zcat command can quickly output their contents to the screen. Use the command with the compressed file's name as the argument as seen below.
[root@bigboy tmp]# zcat /var/log/messages.1.gz ... ... Nov 15 04:08:02 bigboy httpd: httpd shutdown succeeded Nov 15 04:08:04 bigboy httpd: httpd startup succeeded Nov 15 04:08:05 bigboy sendmail[6003]: iACFMLHZ023165: to=<[email protected]>, delay=2+20:45:44, xdelay=00:00:02, mailer=esmtp, pri=6388168, relay=www.clematis4spiders.info. [222.134.66.34], dsn=4.0.0, stat=Deferred: Connection refused by www.clematis4spiders.info. [root@bigboy tmp]#
syslog-ng
The more recent syslog-ng application combines the features of logrotate and syslog to create a much more customizable and feature rich product. This can be easily seen in the discussion of its configuration file that follows.
The /etc/syslog-ng/syslog-ng.conf file
The main configuration file for syslog-ng is the /etc/syslog-ng/sylog-ng.conf file but only rudimentary help on its keywords can be found using the Linux man pages.
[root@zippy tmp]# man syslog-ng.conf
Figure 5-1 has a sample syslog-ng.conf file and outlines some key features. The options section that covers global characteristics is fully commented, but it is the source, destination and log sections that define the true strength of the customizability of syslog-ng.
Figure 5-1 A Sample syslog-ng.conf File
options {
# Number of syslog lines stored in memory before being written to files
sync (0);
# Syslog-ng uses queues
log_fifo_size (1000);
# Create log directories as needed
create_dirs (yes);
# Make the group "logs" own the log files and directories
group (logs);
dir_group (logs);
# Set the file and directory permissions
perm (0640);
dir_perm (0750);
# Check client hostnames for valid DNS characters
check_hostname (yes);
# Specify whether to trust hostname in the log message.
# If "yes", then it is left unchanged, if "no" the server replaces
# it with client's DNS lookup value.
keep_hostname (yes);
# Use DNS fully qualified domain names (FQDN)
# for the names of log file folders
use_fqdn (yes);
use_dns (yes);
# Cache DNS entries for up to 1000 hosts for 12 hours
dns_cache (yes);
dns_cache_size (1000);
dns_cache_expire (43200);
};
# Define all the sources of localhost generated syslog
# messages and label it "d_localhost"
source s_localhost {
pipe ("/proc/kmsg" log_prefix("kernel: "));
unix-stream ("/dev/log");
internal();
};
# Define all the sources of network generated syslog
# messages and label it "d_network"
source s_network {
tcp(max-connections(5000));
udp();
};
# Define the destination "d_localhost" log directory
destination d_localhost {
file ("/var/log/syslog-ng/$YEAR.$MONTH.$DAY/localhost/$FACILITY.log");
};
# Define the destination "d_network" log directory
destination d_network {
file ("/var/log/syslog-ng/$YEAR.$MONTH.$DAY/$HOST/$FACILITY.log");
};
# Any logs that match the "s_localhost" source should be logged
# in the "d_localhost" directory
log { source(s_localhost);
destination(d_localhost);
};
# Any logs that match the "s_network" source should be logged
# in the "d_network" directory
log { source(s_network);
destination(d_network);
};
In our example, the first set of sources is labeled s_localhost. It includes all system messages sent to the Linux /dev/log device, which is one of syslog's data sources, all messages that syslog-ng views as being of an internal nature and additionally inserts the prefix "kernel" to all messages it intercepts on their way to the /proc/kmsg kernel message file.
Unlike a regular syslog server which listens for client messages on UDP port 514, syslog-ng also listens on TCP port 514. The second set of sources is labeled s_network and includes all syslog messages obtained from UDP sources and limits TCP syslog connections to 5000. Limiting the number of connections to help regulate system load is a good practice in the event that some syslog client begins to inundate your server with messages.
Our example also has two destinations for syslog messages, one named d_localhost, the other, d_network. These examples show the flexibility of syslog-ng in using variables. The $YEAR, $MONTH and $DAY variables map to the current year, month and day in YYYY, MM and DD format respectively. Therefore the example:
/var/log/syslog-ng/$YEAR.$MONTH.$DAY/$HOST/$FACILITY.log
refers to a directory called /var/log/syslog-ng/2005.07.09 when messages arrive on July 9, 2005. The $HOST variable refers to the hostname of the syslog client and will map to the client's IP address if DNS services are deactivated in the options section of the syslog-ng.conf file. Similarly the $FACILITY variable refers to the facility of the syslog messages that arrive from that host.
Installing syslog-ng
The most recent syslog-ng and its companion eventlog tar files can be downloaded from the www.balabit.com website. The installation procedure is straightforward, but you will need to have the Linux gcc C programming language compiler preinstalled to be successful. Here are the steps.
1. Download the tar files from the BalaBit website. In this case we have browsed the website beforehand and know the exact URLs to use with the wget command.
[root@zippy tmp]# wget wget http://www.balabit.com/downloads/syslog-ng/2.0/src/eventlog-0.2.5.tar.gz --12:34:17-- wget http://www.balabit.com/downloads/syslog-ng/2.0/src/eventlog-0.2.5.tar.gz => `eventlog-0.2.5.tar.gz' ... ... ... 12:34:19 (162.01 KB/s) - `eventlog-0.2.5.tar.gz' saved [345231] [root@zippy tmp]# wget http://www.balabit.com/downloads/syslog-ng/2.0/src/syslog-ng-2.0.0.tar.gz --12:24:21-- wget http://www.balabit.com/downloads/syslog-ng/2.0/src/syslog-ng-2.0.0.tar.gz => ` syslog-ng-2.0.0.tar.gz' ... ... ... 12:24:24 (156.15 KB/s) - ` syslog-ng-2.0.0.tar.gz' saved [383589] [root@zippy tmp]#
2. Install the prerequisite glib libraries.
[root@zippy tmp]# yum -y install glib
3. Using the tar command we extract the files in the pre-requisite eventlog archive and then use the configure; make and make install commands to install them correctly. Pay special attention to the output of the configure command to make sure that all the pre-installation tests are passed. If not, install the packages the error messages request and then start again.
[root@zippy tmp]# tar -xzf eventlog-0.2.5.tar.gz [root@zippy tmp]# cd eventlog-0.2.5 [root@zippy eventlog-0.2.5]# ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes ... ... ... [root@zippy eventlog-0.2.5]# make Making all in utils make[1]: Entering directory `/tmp/eventlog-0.2.5/utils' sed -e "s,_SCSH_,/usr/bin/scsh," make_class.in >make_class ... ... ... [root@zippy eventlog-0.2.5]# make install Making install in utils make[1]: Entering directory `/tmp/eventlog-0.2.5/utils' make[2]: Entering directory `/tmp/eventlog-0.2.5/utils' ... ... ... make[2]: Leaving directory `/tmp/eventlog-0.2.5' make[1]: Leaving directory `/tmp/eventlog-0.2.5' [root@zippy eventlog-0.2.5]#
4. The next step is to install the prerequisite glib package on your system.
[root@zippy eventlog-0.2.5]# yum -y install glib
5. Some environmental variables also need to be set prior to the installation of the syslog-ng files.
[root@zippy eventlog-0.2.5]# PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/ [root@zippy eventlog-0.2.5]# export PKG_CONFIG_PATH
6. Using the tar command we extract the files in the pre-requisite syslog-ng archive and then use the configure, make clean, make and make install commands to install them correctly. In this case we the --sysconfdir directive with the configure command to make sure syslog-ng searches for its configuration file in the /etc directory. Once again, pay close attention to the pre-installation tests that the configure command executes.
[root@zippy eventlog-0.2.5]# cd /tmp [root@zippy tmp]# tar -xzf syslog-ng-2.0.0.tar.gz [root@zippy tmp]# cd syslog-ng-2.0.0 [root@zippy syslog-ng-2.0.0]# make clean [root@zippy syslog-ng-2.0.0]# ./configure --sysconfdir=/etc checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes ... ... ... [root@zippy syslog-ng-2.0.0]# make; make install Making all in src make[1]: Entering directory `/tmp/ syslog-ng-2.0.0/src' ... ... ... [root@zippy syslog-ng-2.0.0]#
7. The installation has template init.d/syslog-ng scripts and syslog-ng.conf files in the contribs/ directory.
[root@zippy syslog-ng-2.0.0]# ls contrib/ fedora-packaging init.d.RedHat-7.3 init.d.SuSE Makefile.in rhel-packaging syslog-ng.conf.HP-UX syslog-ng.vim init.d.HP-UX init.d.solaris Makefile README syslog2ng init.d.RedHat syslog-ng.conf.RedHat init.d.SunOS Makefile.am relogger.pl syslog-ng.conf.doc syslog-ng.conf.SunOS [root@zippy syslog-ng-2.0.0]#
8. Copy the versions for your operating system to the /etc/init.d and /etc , /etc/logrotate.d , /etc/sysconfig directories. The /etc/syslog-ng/ directory needs to be created beforehand. Redhat and Fedora installations have their own subdirectories contrib/.
[root@zippy syslog-ng-2.0.0]# mkdir /etc/syslog-ng/ [root@zippy syslog-ng-2.0.0]# cp contrib/fedora-packaging/syslog-ng.init \ /etc/init.d/syslog-ng [root@zippy syslog-ng-2.0.0]# cp contrib/fedora-packaging/syslog-ng.conf \ /etc [root@zippy syslog-ng-2.0.0]# cp contrib/fedora-packaging/syslog-ng.sysconfig \ /etc/sysconfig/syslog-ng [root@zippy syslog-ng-2.0.0]# cp contrib/fedora-packaging/syslog-ng.logrotate \ /etc/logrotate.d/syslog-ng
Remember that you may want to customize your syslog-ng.conf file.
9. Change the permissions on your new /etc/inid.d/syslog-ng file.
[root@zippy syslog-ng-2.0.0]# chmod 755 /etc/init.d/syslog-ng
10. You need to be careful. The init.d script may refer to a syslog-ng binary file that's in an incorrect location. Find its true location and edit the script.
[root@zippy syslog-ng-2.0.0]# updatedb [root@zippy syslog-ng-2.0.0]# locate syslog-ng | grep bin /usr/local/sbin/syslog-ng [root@zippy syslog-ng-2.0.0]# vi /etc/init.d/syslog-ng ... #exec="/sbin/syslog-ng" exec="/usr/local/sbin/syslog-ng" ... :wq [root@zippy syslog-ng-2.0.0]#
11. Next create the /etc/syslog-ng directory for the configuration files and the /var/log/syslog-ng directory for the log files.
[root@zippy syslog-ng-2.0.0]# chkconfig syslog off [root@zippy syslog-ng-2.0.0]# chkconfig syslog-ng on [root@zippy syslog-ng-2.0.0]# service syslog stop Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] [root@zippy syslog-ng-2.0.0]# service syslog-ng start syslog-ng: unrecognized service [root@zippy syslog-ng-2.0.0]#
12. The sample syslog-ng.conf file in Figure 5-1 was configured to have all directories owned by the group logs. This user group needs to be created and any users that need access to the directories need to added to this group using the usermod command. In this case the user peter is added to the group and the groups command is used to verify success.
[root@zippy tmp]# groupadd logs [root@zippy tmp]# usermod -G logs peter [root@zippy tmp]# groups peter peter: users logs [root@zippy tmp]# usermod -G logs peter
13. You can now configure syslog-ng to start on the next reboot with the chkconfig command and then use the service command to start it immediately. Remember to stop the old syslog process beforehand.
[root@zippy tmp]# service syslog stop Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] [root@zippy tmp]# chkconfig syslog off [root@zippy tmp]# chkconfig syslog-ng on [root@zippy tmp]# service syslog-ng start Starting system logger: [ OK ] Starting kernel logger: [ OK ] [root@zippy tmp]#
14. Now, your remote hosts should log begin logging to the /var/log/syslog-ng directory. According to our preliminary configuration file, there should be sub-directories categorized by date inside it. Each of these sub-directories in turn will have directories beneath them named after the IP address and/or hostname of the various remote syslog clients and will contain files categorized by syslog facility. In this example we see that the 2005.07.09 directory as received messages from three hosts, 192.168.1.1, 192.168.1.100 and localhost.
[root@zippy tmp]# ls /var/log/syslog-ng/ 2005.07.09 [root@zippy tmp]# ll /var/log/syslog-ng/2005.07.09/ drwxr-x--- 2 root logs 4096 Jul 9 17:01 192-168-1-1.my-web-site.org drwxr-x--- 2 root logs 4096 Jul 9 16:45 192-168-1-99.my-web-site.org drwxr-x--- 2 root logs 4096 Jul 9 23:24 LOGGER [root@zippy tmp]# ls /var/log/syslog-ng/2005.07.09/localhost/ cron.log kern.log local7.log syslog.log [root@zippy tmp]#
Using syslog-ng your system can now be used as a much more customizable tool to help troubleshoot devices attached to your network. Each day syslog-ng will automatically create new sub-directories to match the current date and at the end of each calendar quarter the files will be moved to a special archive directory containing all the data for the previous three months. This archived data can then be periodically deleted as needed. For very large deployments, or for better searching and correlation capabilities, it is possible to send the output of syslog-ng to a SQL type database. This is beyond the scope of this book, but it is a worthwhile feature to keep in mind.
Configuring syslog-ng Clients
Clients logging to the syslog-ng server don't need to have syslog-ng installed on them, a regular syslog client configuration will suffice.
Simple syslog Security
One of the shortcomings of a syslog server is that it doesn't filter out messages from undesirable sources. It is therefore wise to implement the use of TCP wrappers or a firewall to limit the acceptable sources of messages when your server isn't located on a secure network. This will help to limit the effectiveness of syslog based denial of service attacks aimed at filling up your server's hard disk or taxing other system resources that could eventually cause the server to crash.
Remember that regular syslog servers listen on UDP port 514 and syslog-ng servers rely on port 514 for both UDP and TCP. Please refer to Chapter 14, "Linux Firewalls Using iptables", on Linux firewalls for details on how to configure the Linux iptables firewall application and Appendix I, "Miscellaneous Linux Topics", for further information on configuring TCP wrappers.
Conclusion
In the next chapter we cover the installation of Linux applications, and the use of syslog will become increasingly important especially in the troubleshooting of Linux-based firewalls which can be configured to ignore and then log all undesirable packets; the Apache Web server which logs all application programming errors generated by some of the popular scripting languages such as PERL and PHP; and finally, Linux mail whose configuration files are probably the most frequently edited system documents of all and which correspondingly suffer from the most mistakes.
This syslog chapter should make you more confident to learn more about these applications via experimentation because you'll at least know where to look at the first sign of trouble.
