Computational Performance

Increasing Performance

CurveExpert has been optimized to perform regressions and interpolations as quickly as possible. Additional speed gains are sure to some in later versions; however, you can greatly speed up the operation and reliability of CurveExpert by carefully following certain guidelines:

Disable graph preview animations

Right click on the graph preview window (the graphing area in the main window of CurveExpert), and make sure that “Enable Animations” is off. By default, CurveExpert ships with this option enabled in order to provide live feedback on changes being made to the curve fit as it is being calculated.

Scale your datasets

Always, always, always scale your data sets to the order of one. You may do this prior to reading in your data file, or after your data file has been imported using the scaling feature in CurveExpert. This point cannot be overemphasized! If you contact me with a problem concerning a situation where CurveExpert will not fit a certain set of data, this is the first questions that I will ask.

Imagine a data set with x values ranging from 1000 to 10000 and a regression model where the term exp(a*x) is involved. Unless a very good initial guess for a is given, chances are that an exponential to a very large power (eg. exp(1000)) will be taken in the course of the nonlinear regression algorithm. The calculation will overflow, and the regression will fail as a result. Even if the regression happens not to fail, the regression algorithm will have an exceedingly tough time finding the correct parameters, since a small change in the free parameters will cause a tremendous change in the size of the term. The moral of the story: always scale your data! Note that CurveExpert is able to perform correctly on data in any scale, as long as the calculations do not overflow or underflow. So, if a data set is giving problems, scaling it should be the first action to take.

Enable background processing (obsolete)

Enabling background processing (in the File|Preferences dialog) slows down computation considerably, since the calculation must pause for Windows to process pending messages for all running programs. If you are running CurveExpert by itself, or you don’t care if another program is ignored when CurveExpert is calculating, then turn this option off! Calculations speed will increase greatly. The CALC flag in the status bar will be written in red when background processing is disabled to remind you that every other program is halted while CurveExpert works.

Set the tolerance parameter reasonably

Don’t set the tolerance parameter (in the File|Preferences dialog) too low. In regression modeling, not much advantage is to be gained by setting a very strict tolerance. Its main purpose in life is to prevent the nonlinear regression algorithm from converging on local minima.

Use single precision calculations (registered users only)

CurveExpert uses double-precision floating point numbers to do all calculations. The registered version of CurveExpert comes with two executables: one version using single-precision calculations (seven sig figs), and the second using double-precision (15 sig figs). Using a single-precision (theoretically) greatly speeds up performance, at the cost of accuracy. Especially with extremely large data sets, the single-precision version should provide a speed and memory advantage. By registering, you can make the choice yourself. See the sample speed test data below for an indication of the performance of the different packages. You may check what type of data representation is being used by CurveExpert by selecting Help|About.

Avoiding Floating Point Exceptions

CurveExpert takes every precaution to avoid math errors, a.k.a. floating point exceptions. Here are some ways that you can avoid these errors:

Scaling of your data is important

Take care to scale both x and y data to a magnitude of one. For example, if you have data that describes atmospheric pressure at different elevations, you might have (in metric units) a data set that looks like:

x=[-100, 0, 100, 500, 1000, 4000] meters
y=[102000,101325,101000,100500,100000,99900] Pascals

Using the scaling feature in the Data menu of CurveExpert, you can scale this data using a scale factor of 0.001 on the x data, and 0.00001 on the y data. So, you would then have the following data set:

x=[-0.1, 0, 0.1, 0.500, 1, 4] kilometers
y=[1.02,1.01325,1.01,1.005,1.0,0.999] bars

The second example is much more likely to allow nonlinear regressions to converge, and also will allow higher order polynomial fits to be performed with more accuracy. If you have a data set that seems particularly ill-behaved, scaling can help solve this problem.

Data should be appropriate to the model

Make sure that the data is appropriate to the model. Especially look out for using logarithmic or exponential families of models with data that contains zeros or negatives. For example, it is not possible, in any shape or form, to obtain a negative or zero with the basic exponential model (y=ae^(bx), assuming a is nonzero).

So, it is not wise to use a model that cannot reflect the trends in the data. CurveExpert should disallow the use of certain regression models with data that is negative or zero; however, in the case that one should slip through or you are using a user-defined model, you should be aware of the characteristics of the data modeling equations so that choices of an incorrect model can be avoided.

For interpolations, data points must be sorted

If you are using an interpolation-type curve fit, the data points MUST BE SORTED by ascending x! Use the Data|Manipulate feature in CurveExpert to sort your data points correctly. It is quick and painless!

Sometimes, it’s unavoidable

Some curve fits are simply ill-behaved, i.e. prone to divergence. For some data sets, it may not be possible to converge certain nonlinear regression models.