A simulation model’s purpose is to help you understand and mitigate the risks associated with a complex decision, by predicting the future outcomes of different scenarios. A poorly built model introduces risk into an already uncertain scenario, so robustness is key to realising the value of a model. The examples in this article come from our experience in process simulation, but the principles are applicable in all modelling.
A simulation model is a computer program. Behind the screen images, animated display, and user-friendly interface is a lot of computer code. However, if the computer code has an error the model is likely to break at some stage. In the worst cases, the model will be broken inside, but you won’t know it, and will continue to use the results to underpin your decisions. At some stage the error will come to light and if it’s serious you’ll begin to mistrust the model and ultimately the validity of your decisions.
The later you pick up a coding error, the more it will cost to fix. If the error is picked up by the user in the middle of an experimentation program leading into a big decision, this is probably the most embarrassing way for a developer to learn about it. That’s happened to me a couple of times, and it’s not a pleasant experience. The best time to find and fix an error is as early as possible during development.
It’s worth noting that the only thing in a simulation project that is 100% true and up-to-date is the code. True doesn’t mean correct, though! Therefore you owe it to yourself to make sure that the code is correct, and any associated documents (which includes the comments in the code) are an accurate commentary of what’s in the code.
Assuming that you’ve done the basics (e.g. confirmed the business and model requirements with users and other stakeholders, and produced a complete technical specification), here are 10 ideas for model development that will help you code professionally, build an unbreakable model, and avoid introducing “mucho mistrust” into the project.
- Aim for error-free code
- Declare all variables
- Use a naming convention
- Don’t use hard-coded values
- “Assert” values before use
- Use Functions
- Build the model in layers
- Comment the code
- Remove obsolete code
- Re-use proven code
1. Aim for error-free code
This seems like an obvious statement, but there is sometimes a mind-set that “near-enough is good-enough”. This is not the case with what happens inside a simulation model, or indeed any computer program. It only does what it is programmed to do, so what it is programmed to do must be correct. Correctness is the most important consideration. There are some simple approaches and rules of thumb you can use to give you a chance of writing error-free code.
- Keep the code simple. It’s tempting to write code to be as compact as possible, but it’s far better to split a complex one-liner into three or four lines if it makes is easier to see what’s happening. The only negative I can see is that, typically, the more code there is the more chances there are for errors – so make sure every line is error-free!
- Keep the code manageable. On my computer monitor I can see and absorb about 50 lines of code at a time, so I would suggest this is a reasonable maximum length of a block of code. This includes comments, so a block that is more than about 35 lines of executable code is a candidate to be split up.
- Keep the code understandable. Consistent use of indents and comments, meaningful Variable and Function names, and clarity of purpose results in readable code, and if the code can be easily “read” then it is more likely it will be understood.
- Keep the code modular. If you find yourself copy/pasting a block of code a couple of times, this is a sign that you ought to consider creating a Function that reproduces its purpose in a generic way, so that it can be re-used.
- Keep the code test-able. The more modular the code, the easier it is to develop tests to prove the code is accurate. In addition, you should write code assuming that things could be wrong – this is called “defensive coding”, and it forces you to think about how to handle unexpected situations.
2. Declare all variables
Most software languages require you to define each Variable (in a simulation packages this is sometimes called an “Element”) – its name, size and type – before it’s used. If you don’t you are in danger of using a Variable incorrectly, and if you mis-spell its name the software will create a new one “on the fly” and it will probably have a zero value. This will reveal itself at some stage when the model is being tested, usually as a run-time error.
For extra credit:
- If possible, you should declare all integer number Variables as “long” and all real number Variables as “double” to ensure that they can contain both very large and very small numbers, to maximise precision, and to reduce propagating errors caused by rounding
- Set and use the scope of Variables appropriately e.g. In a Function use local Variables for those items that only have meaning inside the Function. If you use a global Variable – one which is available to the whole program – you could inadvertently change a value that is also being used elsewhere.
3. Use a naming convention
A common technique is to use a short prefix (1-3 characters) on a variable/element name so you know the variable type. For example, when using Witness all Machines are prefixed with “m”, all Buffers with “b” and all Conveyors with “c”. Variables are prefixed “i” for integer, “r” for real, “s” for string and “n” for name (i.e. the name of another Variable), and Attributes are prefixed “ai”, “ar”, “as” or “an”. Functions are prefixed “fi”, “fr”, “fs”, “fn” or “fv” which helps you understand the type of the value that the function returns. In this case, “fv” means “void function”, which in Witness-speak means “this function doesn’t return a value”. A void function is known as a Subroutine in other languages.
For extra credit:
- Make each Variable name meaningful
- Do not “re-purpose” the Variable to have more than one meaning
- Where possible refactor the model (i.e. rename the Variables) to keep the names meaningful.
4. Don’t use hard-coded values
Instead use Constants. Hard-coded values are offensive! It’s so easy and more meaningful to use a Constant instead. For example, imagine you have a Variable holding Machine cycle-times called rCycleTimes (10) where the index (1-10) represents the ID associated with each of 10 Machines in the model. If say the fifth machine is called mElevator it would be much clearer and less prone to error to specify the cycle-time as rCycleTimes (iELEVATOR) rather than rCycleTimes (5). All you need to do is define a Constant iELEVATOR and initialise the value to 5. This also means that if you need to extend the model and have another Machine before the Elevator, you simply change the value of IELEVATOR to its new value once – no hunting for where rCycleTimes (5) might be referred in your model.
For extra credit:
- Identify Constants by using all caps in the name
- Initialise all Constants at the same time at the start of the model e.g. by placing them in an initialisation Function.
5. “Assert” values before use
Assertion is common practice amongst software developers, but simulation developers may not be familiar with the technique. The essential idea is that Variables/Elements values are usually set outside the model e.g. in an Excel spreadsheet, and then these values are “read in” by the model. Typically, the range of values that a Variable can take is limited in the model. For example, if a model contains 100 Machines, you may have a variable iMachineNo that can obviously only take the value 1-100. But iMachineNo as a “long” Variable can literally take any integer value between -2,147,483,648 to 2,147,483,647, so in the model you would test that iMachineNo was between 1-100 before it was used.
Remember also that hard-coded values are an abhorrence, so set the number of Machines to a Variable you’ve defined (e.g. iNUMMACHINES) as the data is read-in, and then check that iMachineNo is >0 and <= iNUMMACHINES before is used. This means you detect erroneous values close to where an error exists.
Assertion is not the same as error handling. Assertion is primarily used during development and testing so that you can write better code, whereas error handling is to ensure that run-time errors are presented meaningfully to to user for them to fix.
For extra credit:
- For data set outside the model (e.g. in a spreadsheet), check the values as they are entered by the user
- Assert values in the following situations
- values passed as Parameters to a Function (do the check within the Function)
- values being returned from a Function
- before using a Variable that is an index to an array
- before using a Variable/Element where the value has been determined inside the model
- Develop a technique for enabling/disabling assertion in the model, which you would disable when you deliver the model for use. This is also a reminder that assertion is primarily used during development/testing.
6. Use Functions
A well-constructed model will contain Functions as these enable you to:
- Avoid duplicating code
- Simplify the model
- Examine the model’s internal data
- Make it easier to make changes
For example, if in your model you need to find a Part in a Buffer that matches a set of criteria, it’s just as easy to create a Function that does this for any Part, any Buffer and multiple criteria. You can then test that the Function returns the correct item without having to run the model.
For extra credit:
- Write “Test” functions that call the model Functions with a range of valid and invalid Parameter values to check that the Function is robust.
- Write “Inspect” functions that reports the values of arrays, element states, etc. which you can use as the model is being developed.
- Keep track of how frequently a Function is used by the model – then work to make it as efficient as possible (i.e. as fast as possible) provided always that it remains correct.
7. Build the model in layers
This is really a design point, but it can be useful within the development phase as well. The design should have already separated the model into at least four layers:
- Input data
- Business rules & assumptions
- Process logic
- Output results
Layering allows you to change e.g. the input data, without affecting items within the other layers. The simulation code is part of the process logic layer, and you can sub-layer this to make each component as standalone and therefore as reusable as possible.
A typical example would be how stoppages are applied to Machines. You could specify the stoppage calculations for each Machine individually, but this is likely to lead to code duplication, and a code maintenance nightmare. It’s much better to try and apply stoppages in a standard way. In our models, we use a Function to determine for a Machine the time-between-failures (TBF) and the time-to-repair (TTR) using Parameters such as the Machine ID, and let the Functions take care of the calculation. If we want to change the calculation methodology, we only need to make the change in the TBF and TTR Functions.
For extra credit:
- Use layers for other aspects as well, such as: setups/changeovers; on-screen results display; determining run-length and warm-up duration; and event management.
8. Comment the code
It’s probably accurate to say that most programs don’t have enough comments. Commenting the code doesn’t make it any less efficient, but it will help you and future developers in the months and years ahead. Comments generally should focus on the intent of the code (why and how), not what it does – what the code does should be self-evident from the code itself.
Comments should be inserted wherever it allows the developer to see how code has been structured, whether any special techniques have been used, when and why a change was made, who made the change, what the condition of the model is before and after a block of code has been run, etc. The liberal use of comments together with meaningfully named variables will make the model “self-documenting”.
Whilst comments aren’t part of the code, they should be treated as if they are, and kept accurate. Out-of-date comments will not tell you the truth about the code and will become a source of confusion in the future.
For extra credit:
- Develop a standard comment structure to document every Function. The comments would record each Function’s purpose, parameters (data type and valid values), expected outcome, limitations and assumptions.
- Use comments to keep a change-log for the model as a whole, and also for key sub-components.
9. Remove obsolete code
During a simulation project the model needs to adapt and flex as it generates new insight. This is not an annoyance, it’s actually a sign that the model is delivering value to the business. Whilst you should always try and ensure that the model is backwardly compatible, (i.e. it is able to reproduce identical results from scenarios that have previously been examined using the older process logic), there will come a time when the old logic will never be needed again. It’s tempting to leave the old code within the model and simply comment it out or by-pass the code; however at best this clutters up the model, but more likely it is distracting when you come to maintain that code in the future. If the old code contains a novel technique that you don’t want to forget, “storing” it in the model is not the right place!
For extra credit:
- Remove unused Variables/Elements
- If you have used a “Go To [Label]” or similar statement, check whether you still need the code between the Go To and the Label
- If you have used any conditional statements (e.g. “If…Elseif” or “Select Case” type statements) check that every explicit condition is required, and develop a technique for determining how frequently that block of code is used.
10. Re-use proven code
It’s unusual to build a model from scratch, and sometimes a developer will use another “similar” model to begin with. Unfortunately it’s very rare for an entire model to be a useful starting point. Of course, if the model was built by someone with lower standards than you, there’s always a chance that the model has an error in it that will simply be propagated into the new model. In addition there will be large blocks of code developed for the first model that are of no use in the new one – so this approach generates a lot of obsolete code.
Better instead is to use components that have been extensively tested in previous models and so can be re-used with confidence. At Paragon we have Start-Up models in various software packages that conform to our design strategy, and each contains a library of tried-and-tested components. For each project, if a new or improved component is developed, we update the Start-Up model so it’s available for the next project. An added benefit is that all our models use a standard architectural framework so it’s much easier for a developer to maintain a model that was built by someone else in that framework.
For extra credit:
- Develop Start-Up models which contain your “best practice” techniques, components and design standards
- Develop a post-model review process that keeps your Start-Up models fresh, efficient and relevant.
So there’s my top 10 suggestions. Apply these principles to your model building and you will produce high-quality, confidence inspiring models! What would you do to make sure your model doesn’t have a heart of glass?