A printer-friendly (Adobe Acrobat PDF) version of this document
is also available: Accuracy of the KIC 24-7 VP.pdf (1.28Mb)

Measuring the Accuracy of the KIC 24/7 Virtual Profiles

Philip C. Kazmierowicz - November 16, 2004

Introduction

The KIC^® 24/7™ is a process monitoring system that gives the electronic assembler the ability to monitor the critical parameters of the reflow process and receive immediate notification of any process change. In other words, it's a tool to maintain and control the reflow process.

The purpose of this paper is to measure the accuracy of the 24/7 as it compares to a profile generated with a pass-through thermal profiler.

Profiling the Solder Reflow Process

In a typical solder reflow process, the solder paste company lists a solder paste specification which the engineer may then modify or accept as their process window. The process window consists of a number of individual "statistic specifications": and for each statistic there is specified a target value as well as the upper and lower limit. The technician determines a suitable oven recipe and then a "pass-through" thermal profiling device is used to determine whether or not the profile is within spec. For a complete description of this process, please read the paper online at "Finding the Best Oven Recipe with the SlimKIC 2000 Using the Auto-Focus Option".

Once a suitable oven recipe is found and production begins, the technician must periodically verify that the profile is still within spec. Many companies specify that a pass-through verification profile be run at a stated interval, for instance; once a week, once a day, or every time the oven recipe is changed. The more often verification profiles are run, the more expense is incurred. However, allowing the process to drift out of spec between verification profiles could prove even more costly.

The 24/7 Verification Process

Rather than run a pass-through verification profile at a set interval, the 24/7 automatically calculates the profile and CPK for every production board, along with the means to alarm the user of any process changes. This eliminates the need for verification profiles. Here is a short explanation of the 24/7 system:

The KIC 24/7 measures process temperature along the oven conveyor with 30 thermocouples installed permanently just above the path of the product. In addition the KIC 24/7 independently measures conveyor speed with an encoder that is mounted on a conveyor drive sprocket. The KIC 24/7 records these key process parameters during a pass-through product thermal profile. During production the KIC 24/7 continuously measures process temperature and conveyor speed. As each board exits the oven, the 24/7 computes a "Virtual Profile", based on the changes in process temperature and conveyor speed since the last pass-through profile.

The KIC 24/7 claims to eliminate the need to run regular pass-through verification profiles by recording a Virtual Profile for every production board. The results of each virtual profile are automatically plotted on an SPC chart and a running CpK is calculated. If the CpK drops below 1.3, the process can be stopped and adjusted before any out of spec product is produced.

Above is the KIC 24/7 Charts Tab. In this example there were 111 boards run through the oven. The above screen shows a separate chart for each Statistic. Below shows only the main PWI (Process Window Index™) chart.

In this example we purposefully changed the oven to force a catastrophic failure and then we continued running boards during the failure. We set the 24/7 to warn when the CpK dropped below 1.3. Notice that a CpK warning appeared six boards before the first board was out of spec. This is the power of Statistical Process Control (SPC). The power to predict an out of spec condition before it happens.

For a complete description of the KIC 24/7, please see the online white paper "The KIC 24/7 White Paper".

During production the KIC 24/7 has three operating conditions:

1. Green: The process is in spec with a statistically insignificant chance of going out of spec in the near future (i.e. CpK>1.3). In this case, a pass-through verification profile does not need to be run.
2. Yellow: Process Temperatures have changed significantly and/or there is a statistically significant chance of the process going out of spec in the near future. In this case, a pass-through verification profile must be run.
3. Red: The process is out of spec. Production should be stopped immediately and a pass-through verification profile must be run.

This paper presents the results of a series of tests where we changed the oven loading, oven conveyor speed, oven setpoint temperature, and oven blower speed to test if the KIC 24/7 state always matches the true state of the oven (i.e. the KIC 24/7 is never green when the pass-through thermal profile is out of spec and the KIC 24/7 is never yellow or red without a significant oven change that warrants the running of a verification pass-through profile).

In addition, for each test this paper compares the Virtual Profile with the pass-through profile to measure the accuracy of the Virtual Profile.

Note: The pass-through profile was recorded by attaching thermocouples to various places on an actual PCB and recording the temperature with the SlimKIC^® 2000™ as both the SlimKIC and the PCB pass through the oven. The same PCB was used for each test and the tests were run over the course of several days on a BTU Paragon 150 solder reflow oven.

Getting Started

We chose a board that measures 11.25" by 8.25" and weighs about 190 grams. Five thermocouples were attached to various locations on the board using aluminum tape. A pass-through thermal profile was run using a SlimKIC 2000. A standard leaded solder paste process window was chosen (see table below) and the resulting profile was analyzed to see how well it fit this process window.

The process window used for all profiles in this report

Process Window Index (PWI)

The KIC software includes the ability to represent how well a profile fits the process window with a single number called the "Process Window Index" or "PWI". A PWI of 100% states that the profile is using 100% of the Process Window. A PWI below 100% is considered in-spec. A PWI of 100% or more is considered out of spec. The lower the PWI the better. The PWI allows the quality of two profiles to be easily compared which is useful in this paper where our goal is to determine how well the KIC 24/7 Virtual Profile matches the SlimKIC 2000 pass-through profile.

A PWI is calculated for each thermocouple and each statistic. If the statistic is right in the middle of the
process window, this is perfect and the PWI = 0%. If the statistic is right at the process window limit,
the PWI is 100%. The worst PWI for all the thermocouples and all the statistics is the PWI for the profile.

For a complete description of PWI, please visit the "PWI" section of our web site.

Even if the product thermal profile is in-spec on the first pass, often the profile is on the edge of the process window (i.e. a PWI of 97% is technically in-spec, but any drift in the process is likely to push this out of spec). The KIC software provides an "Oven Recipe Search Engine" option called the KIC^® Navigator™ that automatically determines the oven recipe that will minimize the PWI.

We used the KIC Navigator to find the oven recipe that would minimize the PWI thus optimizing the reflow process. The last pass-through profile run using the optimized oven recipe is called the "baseline profile". With an optimized baseline profile we were ready to begin production.

Above is our baseline profile with a PWI of 32%, which is well inside the specification.

Overview of the Test Results

After setting up the "Baseline Profile" we were ready to start running production. When production starts, the KIC 24/7 records a "virtual profile" for every production board that passes through the oven.

It was now time to make changes to the oven. After each change we allowed the oven to stabilize, then ran a pass-through verification profile. We then compared the PWI of the Virtual Profile to that of the pass-through profile. We also checked the KIC 24/7 "state" (green/yellow/red) with the true "state" of the process as measured by the pass-through profile.

Summary Graph of Test Results

The Tests

The detailed results of each test are presented in the order that they appear in the table on the preceding page. There is approximately one page for each test. At the end of this paper we sum up our conclusions.

Test 1
No Changes

For each Virtual Profile, the KIC 24/7 generates a "Troubleshooting" tab. This is a bar-chart that shows exactly where inside the oven the temperature is different from the baseline profile.

Troubleshooting Tab - No Changes to Oven

Above is the troubleshooting tab for the first verification profile (no changes made to the oven). Each bar represents the temperature change on one of the 30 KICprobe thermocouples that are mounted along the oven conveyor just above the product. The bar on the right shows the conveyor speed deviation as measured by an encoder mounted on the conveyor drive sprocket. The above chart shows that process temperature and conveyor speed in this oven are almost identical to that which was measured during the baseline profile.

Above are the statistics for the Virtual Profile and below are the statistics for the pass-through Profile. Since we did not make any changes to the oven, the statistics are very similar, as expected. It is highly unlikely that any two profiles will ever be identical, so throughout this document, we will characterize two profiles whose statistics are within the measurement accuracy of the SlimKIC 2000, as "equivalent" profiles.

Test #2
Full Oven with Boards 12 inches Apart

This test was designed to determine if loading the oven would have any affect on the profile. The boards are 11.25 inches long and were placed 12 inches apart on the conveyor, so approximately half the usable area in the oven was covered by boards.

The most notable change in the above chart is in the last TC in the last heated zones and the TCs in the cooling zones. Even though all these TC temperatures increased 5+ degrees, the Virtual Profile was not predicting an increase in either reflow temperature or peak temperature:

Above we can see that the results of the pass-through profile match the Virtual profile fairly closely. In fact the differences are within the accuracy rating of the pass-through profiler and we can conclude that the Virtual and Pass-through profiles are equivalent.

Test #3
Full oven with boards 3 inches Apart

For this test we packed the oven even tighter by spacing the boards 3 inches apart. The results were very similar to those when the boards were 12 inches apart.

Limiting the spacing between boards had little effect on the process and the above test concluded that the virtual profile was equivalent to the pass-through profile.

Test #4
Speed Increased by 5%

For this test we increased the conveyor speed by 5%. Note that we had to disable the communication between the KIC 24/7 and the BTU oven; otherwise the KIC 24/7 immediately alarmed stating that the oven recipe was not the same as it was during the baseline profile.

Notice that the "Conveyor Speed Deviation Bar" on the right side of the screen below clearly shows the speed increase.

Below we see that the Virtual Profile is predicting about a 0.5C drop in Peak temp from the
baseline profile and reflow time is predicted to drop about 3 seconds:

The actual drop in peak temperature was closer to 1.0C and the reflow time dropped about 3.5 seconds:

The worst case PWI shifted from "Peak Temp" to "Soak Time" and was predicted perfectly at -51%. The difference between the Virtual Profile and pass-through profiler was within the accuracy rating of the pass-through profiler. Conclusion: When the conveyor speed is increased by 5%, the Virtual Profile provides an equivalent profile to that of the pass-through profiler.

Test #5
Speed Decreased by 5%

For this test the conveyor speed was decreased by 5%.

The pass-through profile statistics show the Virtual Profile to be almost exact. Conclusion: When the conveyor speed is decreased by 5%, the Virtual Profile provides an equivalent profile to that of the pass-through profiler.

Test #6
Speed Increased by 8%

It seemed logical to increase the size of the speed changes to see if the accuracy of the VP would hold up.

As you can see from the above comparison, the results we no less spectacular than on the earlier speed change tests. Conclusion: When the conveyor speed is increased by 8%, the Virtual Profile is equivalent to that recorded by a pass-through profiler.

Test #7
Speed Decreased by 8%

Even with decreasing the conveyor speed by 8%, the VP was still accurately predicting the resulting product thermal profile.

Conclusion: When the conveyor speed is decreased by 8%, the Virtual Profile provides an equivalent profile to that of the pass-through profiler.

We considered larger speed changes, but the KIC 24/7 software is setup to always alarm if the conveyor speed deviates by more than plus or minus 10%. This alarm will occur, even if the Virtual Profile PWI is not out of spec:

This alarm should strongly discourage the user from continuing production without first running another pass-through thermal profile. Conclusion: For larger changes in the conveyor speed, the 24/7 will automatically warn the user to check the process.

Test #8
Increase Zone 9 and 10 by 10 degrees C

For the following four tests we changed the temperature in both zone 9 and zone 10.

Above you can clearly see that the temperature in zones 9 and 10 is up about 10 degrees from the baseline.

The new PWI is 79% (up from 33% in the Baseline Profile). The Virtual profile did an excellent job of predicting this change, with the average difference between the Virtual peak temperatures and the actual peak temperatures less than 0.5C.

It is important to note that even the relatively small change of only 10 degrees immediately brought on the "Process Temperatures Have Change Significantly" warning:

This warning lets the user know that significant changes in process temperature have occurred, even if those changes are not currently pushing the profile out of spec.

Conclusion:

1. When changing two zones by 10%, the 24/7 Virtual Profile will be equivalent to a pass-through thermal profile.
2. The KIC 24/7 will notify the user when oven temperatures have changed significantly, even if the profile remains within spec.
3. The KIC 24/7 will show precisely where the change inside the oven has occurred. Often the pass-through profile will indicate that the oven has changed, but it is not nearly as obvious where inside the oven the changes have occurred.

Test 9
Decrease Zone 9 and 10 by 10 degrees C

Dropping the zone 9 and 10 temperatures is just as obvious on the troubleshooting tab.

Here the difference between the VP peak temp and the actual was an average of 1.7C. Note: The 10 degree setpoint decrease of this test set off the warning described on the previous page. For simplicity, we chose not to show this warning for each experiment.

Conclusions:

1. The Accuracy of the Virtual Profile was well within an acceptable and useful range.
2. The 24/7 will alarm the user when the oven parameters have changed significantly, even if the pass-through profile remains within spec.

Test 10
Increase Zone 9 and 10 by 20 degrees C

The following two tests change zones 9 and 10 by 20 degrees each.

The VP predicted all the TCs to within about 1 degree except TC #8 which was 2.5 degrees. It is possible that TC#8 came a bit loose on this profile. It is important to note that the overall PWI is always the worst one. In this case the worst case PWI is off by 24%, but all the others are off by no more than 11%. Since the 10 degree setpoint change for Tests 8 & 9 set off the "large temperature variance" alarm, of course the 20 degree temperature change also did. For simplicity, we chose not to show the alarm for each experiment.

Conclusions:

1. Even though there is a significant change in oven setpoints, the Virtual Profile worst case scenario varies by 25%. Yet that 25% is a difference in temperatures of only 2.5C.
2. The large change in oven temperature would set off an alarm instructing the user to run another profile immediately.

Test 11
Decrease Zone 9 and 10 by 20 degrees C

In this test the VP PWI only increased to 89% which is still in spec, however, the pass-through product thermal profile PWI was 104%, which is just outside the spec limits. The difference between temperatures of the virtual and the pass-through product is 1.5 degrees. As mentioned earlier, the KIC 24/7 also displayed a warning which would have directed the operator to run another profile before continuing production.

Also, one glance at the troubleshooting tab would have made it clear, even to a novice, that this process has changed significantly since the baseline profile and all the change is in zones 9 & 10.

Conclusions:

1. When lowering setpoints by 20 degrees, the largest PWI variance was 15%, or 1.5 degrees C.
2. When temperature variations occur over time within the oven, the Virtual profile will alarm the user and identify exactly where inside the oven the temperature has changed.

Test 12
Disable Zone 10 Top Blower

In this test we physically unplugged the blower motor at the top of zone 10. The oven control PC did not give us any alarm. However, the troubleshooting tab clearly shows something strange going on in zone 10.

As you can see below, the VP predicted that the process would stay well in spec and this was verified by the pass-through thermal profile.

While disabling the top blower in Zone 10 did not have a significant affect on either the Virtual Profile PWI or the pass-through PWI, the KIC 24/7 did notify the user with the following warning:

This warning lets the user know immediately that there is a problem with the oven and the Troubleshooting Tab identifies exactly where the problem is inside the oven.

Conclusions:

1. When there was a decrease in the convection rate, the oven was able to maintain the profile, and the 24/7 Virtual Profile was equivalent to the pass-through profile.
2. The 24/7 was able to warn the user that the process had changed, and identify precisely where inside the oven the change occurred, whereas the pass-through profile gave no warning of process change.

Test 13
Disable Zone 10 Bottom Blower:

The results of this test were almost identical to those of the previous test, but the changes in zone 10 on the Troubleshooting Tab were even more pronounced.

Just like in the previous test, the VP and Pass-through profile statistics were almost unchanged, and the "Process Temperatures Have Significantly Changed" warning again appeared on the 24/7.

Conclusions:

1. When there was a decrease in the convection rate, the oven was able to maintain the profile, and the 24/7 Virtual Profile was the same as that of the pass-through profile.
2. Although the 24/7 was able to warn the user that the process had changed, the pass-through profile gave no warning of process change.

Observations

1. Our tests show that with oven zone changes of less than 10 degrees, the accuracy of the 24/7 "Virtual Profile" is equivalent to the profile obtained through the use of a pass-through profiler.
2. When oven temperature changes of greater than 10 degrees occur, the Virtual Profile is still very accurate, although not perfect. However, during these events, the 24/7 will alarm, notifying the user to check the profile.
3. There are oven events (such as disabled blowers) for which a pass-through profiler does not generate a warning. However, these events may have a significant impact on the thermal process. In these instances, the 24/7 warned of a problem while the pass-through profiler did not.
4. The 24/7 will detect changes to the thermal process even when the change has no immediate effect on the profile. Such changes can be remedied before the process drifts out of spec, a requirement for zero-defect production.
5. The 24/7 troubleshooting screen immediately indicates to the operator where inside the oven any temperature changes have occurred.

Wrap-up

The purpose of this paper was to measure the accuracy of the 24/7 as it compares to a profile generated with a pass-through profiler.

During production the KIC 24/7 has three operating conditions:

1. Green: The process is in spec with a statistically insignificant chance of going out of spec in the near future (i.e. CpK>1.3). In this case, a verification profile does not need to be run.
2. Yellow: Process Temperatures have changed significantly and/or there is a statistically significant chance of the process going out of spec in the near future. In this case, a verification profile must be run.
3. Red: The process is out of spec. Production should be stopped immediately and a verification profile must be run.

In this paper we presented the results of a series of tests where we changed the oven loading, oven conveyor speed, oven setpoint temperature, and oven blower speed. The results of these tests verified that the state of the KIC 24/7 always matches the true state of the oven. (i.e. the KIC 24/7 is never green when the product thermal profile is out of spec and the KIC 24/7 is never yellow or red without a significant oven change that warrants the running of a verification thermal profile).

We determined that for small process changes, the 24/7 "Virtual Profile" provides an equivalent profile to that of a pass-through profiler. For larger process changes, the 24/7 Virtual Profile did not exactly match the pass-through profile, however, in every case the 24/7 immediately warned the user that the oven had changed significantly and that a pass-through profile should be run immediately.

We also discovered that there are situations where significant thermal process changes occur that have no immediate affect on the quality of the pass-through profile. In these situations, the pass-through profiler did not warn the user of the changes in the oven. However the KIC 24/7 did immediately warn the user that something had changed within the oven, and pinpointed exactly where the change had occurred.

Conclusion

The KIC 24/7 is a powerful tool that accurately, continuously, and automatically monitors the solder reflow process. The KIC 24/7 records a "Virtual Profile" for every production board (which this paper has shown to be equivalent to the profile from a pass-through thermal profiler). The KIC 24/7 will identify significant thermal process changes immediately, even those that do not affect the quality of the profile and are thus difficult to track with a pass-through thermal profiler. The KIC 24/7 will identify trends toward failure long before actual product failure occurs.

Company || PWI || Products || Ordering || Press || Support || Download || Library
Home || Contact || Links || Search

Copyright (C) KIC. All rights reserved.
A Division of Embedded Designs Inc.