yapfr
What it is:
yapfr is a Linux specific, console based, system performance monitor (/proc
file reader). It is designed to provide a quick means to evaluate performance
and find bottlenecks on systems.
Some notable features of yapfr are:
- Graphical memory, disk, network, load, and CPU usage representation.
- Top-n processes by memory usage, page faults, open files, or CPU usage.
- Graphical CPU utilization supports SMP systems
- Network utilization and TCP connection information
- Disk I/O - utilization.
- System V statistics.
- Kernel statistics
- Interrupt statistics (for up to 8 processor systems)
- User mode - lists users logged into the system.
- Completely (curses) character based.
- Warning thresholds for various memory related parameters
- Duration graphical load and CPU monitoring
- Runlevel & kernel version reporting
- Paging activity
Design Philosophy:
If any can be claimed, it is to:
- Create a tool that can be quickly referenced to find the most
fundamental stats of a machine.
- Develop a one-point tool for multiple statistics that does not require the
user to remember multiple commands or options for each statistic.
- Brush up on my Linux / Unix fundamentals. In my opinion, you learn the
most about something by writing code for it.
- Develop a performance monitoring tool that is NATIVE TO LINUX and is not
just a port of some tool popular for another platform. (One such example was a
"big-Unix" performance monitoring tool that uses the same CPU
measurements as found in the native tools (and API) from the
"big-Unix" it was ported from. The problem was that these values did
not map to the values provided by the Linux /proc file system. So depending
upon the Linux kernel you were running on you were either discarding (or
summarizing) data or were generating values that were not explicitly provided
by those (earlier) kernels.)
As for the name... I figure the Unix world needs yet another program that begins with ya.
Screen-shots:
|
This is a RHEL 4 system running in a vmware session. yapfr is in "disk graph" mode. VMWare is an excellent means of testing against multiple Linux distributions.
|
+--------------------------+-----------------+--Runlevel: N 5---+--V: 2.0.1--+
| lnx408 |14:57:30 05/20/03| up 0d 23h57m52s| 2 Users |
+--------------------------+-----------------+---------------{2.4.19-64GB-SMP}
| User 20.8 % [#### ] System 4.8 % [ ] |
| Idle 74.5 % [############## ] Nice 0.0 % [ ] |
+----------------------------------------------------------------------------+
| Load: 1: 0.89 5: 0.74 15: 0.39 [ ] |
+----------------------------------------------------------------------------+
| CPU3 User 44% [######### ] Idle 49% [########## ] |
| System 7% [# ] Nice 0% [ ] |
| CPU2 User 25% [##### ] Idle 64% [############# ] |
| System 11% [## ] Nice 0% [ ] |
| CPU1 User 1% [ ] Idle 99% [#################### ] |
| System 0% [ ] Nice 0% [ ] |
| CPU0 User 13% [## ] Idle 86% [################## ] |
| System 1% [ ] Nice 0% [ ] |
+----------------------------------------------------------------------------+
|
| This is a four processor system running in CPU mode running SuSE Enterprise 8. |
+--------------------------+-----------------+--Runlevel: N 3---+--V: 3.17.0-+
| paris.favorite.com |12:19:32 10/11/05| up 0d 19h14m30s| 10 Users |
+--------------------------+-----------------+-------------{2.6.11-1.1369_FC4}
| User 1.0 % [ ] System 0.0 % [ ] |
| Idle 99.0 % [################### ] Nice 0.0 % [ ] |
| Wait: 0.0 % [ ] Interrupts Hard: 0.0 % Soft: 0.0 % |
+----------------------------------------------------------------------------+
| Load: 1: 0.00 5: 0.00 15: 0.00 [ ] |
+----------------------------------------------------------------------------+
| CPU0 User 1% [ ] Idle 99% [#################### ] |
| System 0% [ ] Nice 0% [ ] |
| Speed: 1797.516 BogoMIPS: 3555.32 Cache: 256 KB HyperThreading: N |
+----------------------------------------------------------------------------+
|
| This is a single processor system running in CPU mode running Fedora Core 4. |
+--------------------------+-----------------+--Runlevel: N 3---+--V: 2.0.2--+
| lnx408 |15:03:36 05/21/03| up 0d 19h 7m13s| 6 Users |
+--------------------------+-----------------+---------------{2.4.19-64GB-SMP}
| User 0.2 % [ ] System 0.0 % [ ] |
| Idle 99.8 % [################### ] Nice 0.0 % [ ] |
+----------------------------------------------------------------------------+
| Load: 1: 0.85 5: 0.35 15: 0.13 [ ] |
+----------------------------------------------------------------------------+
| Mount Space________%___ Inodes_________________%___ |
|_Point________________________Total_Avail__Used_Total_______Avail______Used_|
| / 3 G 3 G 16% 4294967295 4294967295 0% |
| [########### ] |
| /boot 104 M 65 M 37% 4294967295 4294967295 0% |
| [######################### ] |
| /home 4 G 4 G 1% 4294967295 4294967295 0% |
| [ ] |
| /usr 8 G 6 G 20% 4294967295 4294967295 0% |
| [############## ] |
| /var 2 G 1 G 27% 4294967295 4294967295 0% |
| [################### ] |
| /fs/reiser 4 G 3 G 33% 4294967295 4294967295 0% |
| [####################### ] |
| /fs/xfs 4 G 1 G 66% 4198272 3882168 8% |
| [############################################## ] |
| /fs/jfs 4 G 1 G 67% 3096736 2772101 10% |
| [############################################## ] |
| /fs/ext3 4 G 1 G 65% 524832 189538 64% |
| [############################################# ] |
+----------------------------------------------------------------------------+
|
| This system is running mostly Reiser file systems (note the inode counts) except for the bottom few filesystems. An interesting note: The bottom four filesystems hold the same number of files (approx 1/4 Million). |
|
This is my workstation running in "memory" mode.
|
|
This is my workstation running in "load" mode.
|
|
This is my workstation running in "partition" mode.
|
The Distribution:
The source:
The distribution is source only. It is tested against multiple distributions on multiple architectures, but is not maintained as a package for each.
Development: yapfr-4_3_0.tar.gz
Production: yapfr-4_2_13.tar.gz
Recommended: yapfr-4_2_14.tar.gz
Latest Development releases:
yapfr-4_3_0.tar.gz
- Option to filter out (redundant) partition data in y, D, and P modes.
- Fix for reading ~/.yapfrrc
- Fix for uninitialized string.
- Code documentation and update to yapfr.pdf
yapfr-4_2_14.tar.gz
- Fix for garbage in (non-terminated) device names.
yapfr-4_2_13.tar.gz
- Debug of latest development.
- No RPM for this version - despite it is the recommended version.
- J mode(s) should be done.
yapfr-4_2_9.tar.gz
- Port to (verified on) Red Hat RHEL 5.
- Now availible as an RPM (for RHEL 5 x86 (NOT x86_64 or PPC) - but should work elsewhere on x86)
- Now includes j and J modes. j is under development (will be complete in 4.3.0) and J is fairly solid
- j - vm stats mode
- J - vm page in page out graph mode
The code is in C++ and requires the standard tools and utilities to build. It compiles against ncurses.h and links against the curses libraries. For this reason, the (n)curses development packages must be installed. The documentation
is written in LaTeX and requires pdflatex to compile.
If your system does not include the TeX tools then you can run make with the target of
"yapfr" instead of just "make". The pdf documentation is available online
here for those without the TeX tools.
This code is released under the GPL.
All older versions have been removed. The feature-set, stability, and accuracy of the
latest version is significantly better than the older versions.
Development Notes:
- Some of the code in yapfr is over six years old with well over 100
distinct versions, and nearly 50 source (.cpp) files. - The code layout in
the original design has not been perfect, but has been maintainable through
significant changes in features and language / library usage. The project
management of all my subsequent projects has been influenced by the design
of this one. Some of the key findings have been:
- Strong versioning scheme. This is nothing new, but I have found it
invaluable in keeping development straight on as many as 4 systems
at once.
- Clear version change log and to-do list. This is maintained in the
version.h file. This file is over twice the size of any other source
file in the project and is the first to be opened on a change and
the last to be closed.
- I rarely have properly recognized code reuse (in the proper sense)
from project to project. (I believe this is from not being a full
time developer on a consistent problem domain). I always end up
cutting and pasting rather than building common libraries. But...
If the code is properly segregated and documented, it is easier to
find and reuse (both in the project and when I go looking for it
from another project).
- I enjoy writing in C++, but prefer C for binary portability. The single
largest issue with distributing yapfr is that it must be compiled for the
platform you are on. Because of my experience with yapfr, I do not consider
C++ as a viable language for Linux development (given the requirement to
distribute a single binary.) All my (similar) projects since yapfr have been
in C.
Archived Versions:
yapfr-4_2_2.tar.gz
- This is the first port to Ubuntu. (I am a late adopter, and still just trying it out). Fortunately the port turned up some minor issues that could have effected the Red Hat ports. Installing (compiling) on Ubuntu requires the installation of the ncurses library (apt-get install libncurses5-dev).
- Some testing was done on Oracle Enterprise Linux (a Red Hat variant with no evident issues. The changes in 4.2.2 should not have an impact on this port.
yapfr-4_2_1.tar.gz
- I added an extra line to the "G" mode that sums all RSS values for all processes. Granted, it has no (current / easy) way of subtracting shared (resident) code from this number, so it is not that accurate. But, I was growing tired of the stats for memory usage that were obviously wrong. I was using the same values that show up in other utilities (we all get our data from the same place). I just wanted another number to look at to give some indication of actual memory usage.
yapfr-4_1_2.tar.gz
- A segfault was found and fixed in the Fedora Core 5 port. (It was pre-existing, but found while running on that version).
yapfr-4_1_0.tar.gz
- 4.0 was snapped once the disk graph mode was solid. 4.1 was created shortly afterward to add additional makefile targets that did not require the install to build the pdf file. The pdf file is available here.
yapfr-3_27_0.tar.gz
- Fixes to core API's to fix issues caused by bad input on AMD 64 kernels.
- Added a "y" mode that displays graphical disk utilization.
- Note: yapfr must run as root or setuid root to have accurate statistics for this mode. The code to find the sector size requires root access to open() the device before calling ioctl().
yapfr-3_17_0.tar.gz
- Windows now resize properly. I have noticed a bug in x86-64 that causes errors when resizing. I am not sure that this is a yapfr problem as I have seen it with top and other curses based apps as well.
- Moved device name resolution to a central class that handles (device name) resolution regardless of where it is found (/sys or /proc).
- Added a 'z' mode that graphs CPU utilization over time.
- Tested on a number of platforms from Red Hat 6.2 (a 2.2 kernel) to Fedora Core 4 (a 2.6.11 kernel).
- I have noted that the phrase "Code Cleanup& appears repeatedly in each version. It is particularly true in this release as I was changing code that was written sometime in early 2002. This project is that kind of monster.
yapfr-3_10_0.tar.gz (Tested on: SuSE 9 PPC & x86, RHEL 3/4 PPC & x86)
- Entirely curses based. The non-curses approach had a number of ugly drawbacks in maintaining multiple child processes. Since it was largely curses based anyways, I converted the last to curses and now it responds better (properly) under load.
- CPU, and other modes now read 2.6 statistics. This was highly relevant for CPU stats.
- Now supports /sys filesystem.
- Fixed a number of PPC related issues.
- Fixed memory display issue in memory modes.
- SysV mode now toggles between stats with an additional press of the 5 key.
- Last major changes prior to 4.0 release. (I might add a CPU graph mode - Still undecided).
yapfr-3_6_5.tar.gz (Tested on Red Hat Power PC port and Red Hat AS 4)
- Fixed problem with the array used in the load graph mode. (This problem only manifested itself on Red Hat AS 4 on AMD 64 when running in curses mode. Simple problem, ugly symptoms, difficult debug, easy fix.)
- Ported to Power5 architecture. Addressed some enndian issues (my fault) and some /proc file format issues (ppc64 port's fault).
yapfr-3_6_2.tar.gz (2.6 compatible)
- Fixed issue with reporting the process in TCP mode when the process had exited.
yapfr-3_6_1.tar.gz (2.6 compatible)
- More code cleanup
- More accuracy in "F" mode.
- Bounds checking and more accurate rounding in some modes.
- Added socket info to default screen.
- Now forks (in default mode) and counts sockets. If the number is high, it will time out and not display socket info for the default mode.
yapfr-2_28_3.tar.gz (2.6 compatible)
- Some of the class inheritance was cleaned up. The class "lineage" is much cleaner now.
- You can filter in or filter out processes by owner in the various lists.
- An additional "lightweight" default mode was added.
- Documentation reflects 3.0 feature set.
- Support (or handling of) IPv6 addresses in T and t modes. (They are not printed (just not enough room) but they are handled like IPv4 addresses otherwise.)
- Added MAX_PROC_LIST_QUIT & MAX_PROC_LIST_READ (as well as TCP list equivalents) that prevent yapfr from working too hard when there is an excessive number of processes or socket connections. (On systems with greater than 2000 to 3000 processes (or threads) yapfr can appear to hang while collecting that much data. This prevents that.)
- Alot of code clean-up and documentation.
- Support for EMC major numbers (EMC PowerPath pseudo devices). Tested against EMC DMX volumes.
yapfr-2_14_0.tar.gz (2.6 compatible)
- This version is based on changes to the "Partition" statistics only.
- The data is more accurate and there are more fields of information.
- Additional entries were made for EMC PowerPath (pseudo-)devices.
- This is the only version of yapfr that will gather disk statistics from a 2.6 kernel. Currently it uses /proc/diskstats. There are future plans to look for a sysfs file system if it has been mounted.
- The "D" (Disk) mode will not work with a 2.6 kernel. This mode has always displayed a "No disks found" message when disks were not found in /proc/stat. (This is a common occurrence with the /dev/cciss/cxdy devices from the Compaq controllers.) So this mode will always show no disks found when run on a 2.6 kernel.
- Good news / bad news: Some changes were made to the Partition display code that show
numbers more accurately. Unfortunately this is of little help to view these stats from
most 2.4 kernels (/proc/partitions) that I have tested with. The numbers from 2.6.0
(/proc/diskstats) appear much more reliable.
yapfr-2_9_11.tar.gz
- The changes from 2.9.2 are largely code clean-up and internal issues, but does include some formatting and usage fixes.
- Documentation still lags development. I am waiting to finish up all 3.0 features before I re-visit the documentation.
- curses support is on by default, and still has some (rare) resizing issues. Remember this as you use it.
- One of the biggest changes is a single file buffer method. I created a single buffer for all file buffering. This required the creation of an internal semaphore to limit concurrent access to the buffer.
yapfr-2_9_2.tar.gz
- It has fixes and enhancements to 2.5.1
- It has at least one new mode.
- Documentation is still at around 2.5.1
- It now uses curses for much improved screen refresh. Set USE_CURSES=0 in the Makefile to disable use of the curses library / functions.
- There is one known issue with screen resizing; If a screen is resized (such as when in an XTerm) yapfr *may* die. This tends to only happen after multiple resizings.
- As before, read the version.h file in the source for the most complete commentary on the code.
yapfr-2_5_1.tar.gz
- It has at 5 completely new modes.
- It fixes some ugly (glaring) problems with d and D modes.
- It resolves a few bugs in the production release (2.0.2).
- It changes a number of command line options and how modes work.
- Inter-application help (h mode, --help command line option) are up to date, but the accompanying documentation has yet to be updated.
- There are so many significant changes between 2.0 and 2.5 that I expect to go to 3.0 soon without many more changes. At this point I am looking at minor feature changes and mostly documentation, clean-up and testing before the 3.0 release.
- Read the version.h file in the source for the most complete commentary on the code.