Find out who has a stake in the success (or failure) of a project and
get them (or their representatives) involved in discussing possible solutions.
People tend to prefer systems that give them something... They also
like system that they have helped to develop. So getting stakeholders
involved will improve the project.
They will also improve the chances of its
environment accepting it.
Once there was a tall office building with a set of elevators.
People always were complaining that they were too slow.
They would have to wait
when they entered the building before the elevator came.
Exercise in class: What are the problems? Propose solutions.
Moral: TBA in class.
These are a little abstract but might help you come up with an
innovative proposal to fix a broken (or bent) system:
[ Twelve_leverage_points ]
(Wikipedia).
Study this
[ world-us-canada-11491937 ]
news item.
Moral: always be careful to work out how much data a system can handle and make sure
this is more than the projected amount.
- Situation->-THINK* ->-Ideas* ->-Business Case->-Preliminary report . . .
- * Involve stakeholders.
- Senior Managers
- Your Boss
- Your users.
-- involve opinion leaders and users who are expert in the area,
for preference.
- Customers
- Other Developers
- ...
- Notice that the process is iterative. You may have to make several
proposals until you can proceed to detailed design and implementation.
The Business case for a project explains the
Business Reasons
for carrying out the project. Funcdamentally all businesses
have two reasons to carry out a project:
- Increase Revenue
- Decrease Costs
Luckily some objectives tend to do both: improving reliability,
improving efficiency, reducing energy use, ... But there
is often some up-front costs that have to be met before the
project starts to pay off. There is a standard approch for handling
the tradeoff between development costs and ultimate benefits... it
is call "Cost Benefit Analysis" and will be described later.
By the way -- increasing revenue is the real selling point for a project -- see
[Armour11]
for example.
First you need to understand the situation -- and the previous
activities and notes are all about gathering and organizing the
information that you need to come up with good ideas.
It helps if you train your brain to follow some workable patterns below
A good pattern for thinking is TEC from the British Cognitive
Research Trust.
TEC is the acronym for: Target Expand Contract.
- TEC::CoRT=following,
- Pick a specific Target.
- Expand about this target -- say everything you can.
- Contract to a selection, summary, solution, conclusion, action, or ....
(After the Cognitive Research Trust "THINKING -- ACTION" booklet, UK, 1975?)
Here is another useful pattern. When evauating an idea start by looking for the positive points and end by looking for things you don't know.
- Evaluating_an_idea::=following
- + Good points
- - Bad Points
- ! Interesting points.
- ? Unknowns
The best way to solve a problem is to avoid having it in the first place.
Be aware that an important job when analyzing and designing systems
is to avoid problems rather than to create and then solve them.
Choosing your battles is a significant part of successful systems work.
In many cases, the correct description of the problem contains
the solution to the problem In other cases explaining what the problem is
-- verbally -- or better on paper -- helps you create or discover the
solution. For example see
[ rubber-duck-problem-solving.html ]
Also see Problem_solving and Polya below.
Sometimes our task is to solve a problem that exists in the current system.
Problem solving is a skill that you can learn. It pays to practice it.
I've put a set of online resources that you can refer to later at the end
of this section.
Here is a procedure I've used for many decades
- Problem_solving::=following
- Clarify what you really want -- the goal or purpose.
- Review the given data and unknowns.
- Get data to reduce the unknowns.
- Generate at least three solutions
- Do nothing
- The obvious solution
- If X was the solution then ...
- Shift your goal a little...
- Factorize the problem into subproblems, solve them
( Problem_solving (recursive)), and assemble combinations.
- Use a random thought to stimulate an idea. Example open a dictionary at
random and pick a random word, now try to find a connection between the
word and the problem.
- Re-express the problem in a different media or mode. Typically this means
drawing a different kind of diagram, but writing a poem on the topic
can also generate insight and so solutions.
Check out
Visual Thinking
[ http://www.wired.com/magazine/2010/09/st_thompson_visual/ ]
[ http://www.thebackofthenapkin.com/ ]
(advert).
- ...
- Improve any of the solutions that you've got
- Evaluate the solutions. Since each solution should be evaluated on several
criteria it is beset to draw up a table. I use rows for the criteria and
columns for the solution.
For example with personal projects I use:
Table| Criteria | Solution1 | Solution2 | Solution3 | ...
|
|---|
| Feelings | ? | ? | ?
|
| Ease | ? | ? | ?
|
| Cost | ? | ? | ?
|
| fit with Goals | ? | ? | ?
|
| Best case | ? | ? | ?
|
| Worst case | ? | ? | ?
|
| How handle worst | ? | ? | ?
|
| ...
|
(Close Table)
In an enterprise use a table something like this:
Table| Criteria | Solution1 | Solution2 | Solution3
|
|---|
| Benefit | ? | ? | ?
|
| Cost | ? | ? | ?
|
| Risk1 | ? | ? | ?
|
| ...
|
(Close Table)
- Choose a solution. Why that solution?
- Make a plan of action including: NOW! Development, and Support...
A colleague of mine with a lot of experience said that every problem
in an organization needed three solutions. Here I've translated them into
a medical analogy:
- Emergency first aid -- a fast patch to stop the bleeding and keep the system breathing... Also called the band-aid solution.
- Curing the underlying disease or repairing the damaged body.
- Installing healthy habits to stop the disease re-occurring.
If the problem was a buffer overflow letting
people take over the server.... then the band aid is to shutdown
and clean the system. The cure is to find the buffer overrun
in the code and fix it. The healthy habit is to give up
'fgets' and C.
Some problems can be expressed in terms of Givens, Goals, and Operations
that relate them. These can often be solved using the mathematical method as described by George Polya. The following may also help you in mathematics classes.
- Princeton 1988 2nd edn isbn0-691-02356-5 QA11P6 1988
[Polya88]
1. Understand, 2. Plan, 3. Carry_out_the_plan, 4. Looking_Back.
- What is the unknown? -- goal/output/conclusion?
- What are the data/givens? -- given/input/parameters/hypothesis
- What are the conditions? -- operations/relations/premises connecting the
given to goal.
- Check conditions: possible & sufficient ~ redundant ~ contradictory ~ separate.
- Draw a diagram.
- Introduce a suitable notation.
- Separate the various parts of the condition. Can you write them down?
- Use plausible, reversed, and lateral thinking to invent a way to solve the problem. Avoid rigor and details.
- Look at the goal/end point first!
- Have you seen a similar problem before?
- Look for similar goal/given/...
- Look for more general/specific.
- Look at any previously solved problem/theorem... can you make use of it?
- Can you restate/reverse this problem?
- Review
- Did you use all the data/givens?
- did you use the whole condition?
- Have you allowed all the notions in the problem... if not revise your plan.
- Carry out your plan step by rigorous step.
- Fill in the details in steps top-down by refinement.
- At each step look for signs of progress,
- Check each step for correctness.
- Can you see that it is correct(intuitive)?
- Use formal techniques that fill in the details in your plan.
- Can you prove that it is correct?
- Can you check the result?
- Can you check the steps?
- Did you use all the givens?
- Can you improve the steps?
- Can you see the whole at a glance?
- Can you use the result or the method in some other problem?
- How can you share/present/publish the result?
. . . . . . . . . ( end of section Polya) <<Contents | End>>
How to spot something that might be worth changing.
- Requests for changes(RFCs). Does your enterprise have a form for systems
requests? Should it? Does a committee prioritize them? Map out the
request handling system as a DFD!
- Look at the Strategic plan -- if it exists.
- Who is responsible for deciding on projects. Some enterprises
have a special
Steering Committee
for IT projects.
- List the stakeholders:
- People who have something to lose if the project or system fails.
- stake_holders::="people who have a stake in the success of the system".
- stake_holder::="person who has a stake in the success of the system".
- Look at Possible Futures: Best, most likely, and worst.
What can you do about the worst case scenarios?
- Look for the forces and barriers in the situation.
- The 4 Reasons for change:
PORT
- Problems -- where does it hurt? (just like medical diagnosis)
- Opportunities
- Risks -- from inside and boundaries.
- Threats -- From outside
- Some Reasons for
Systems Requests
- to improve service
- to support new products/services
- to deliver new or better information
- to strengthen control and security of data
- to reduce cost
- etc.
- Costs and Benefits: Before a project can proceed you will need to convince people that it
is worth doing.
This often means you need to prove that
it will either save money or generate more money.
Note that most businesses aim to optimize profit: (minimize
costs and maximize income).
However some enterprises (CSUSB and non-profits
for instant) treat money as a constraint and have
some other objective. You need to know what the
organization's vision, mission, goals, etc.
are before proposing a problem to tackle. We will return to
formal Cost-Benefit-Analysis later in this course.
- Half-a-dozen factors that effect projects
- Internal
- Existing systems
- Strategic planning
- Top management (stake_holder)
- users (stake_holder)
- People in the IT Department (stake_holder)
- External
- Technology (Opportunity / Threat)
- Suppliers (stake_holder?)
- Customers (stake_holder?)
- Competitors (Threat / opportunity)
- The Economy (Threat / opportunity)
- Governments (Threat / opportunity)
This pattern is named for a movie where the central character is told
that "If he builds it, they will come".
This is a plan that is most often used with
commercial and mass marketed software. You develop the new system so that
you think it has clear advantages over the old (rival) system and then
encourage people to use the new one in place of the old. Given that the
new system is attractive enough to make people change then the numbers of
people switching over follows a bell shaped curve with small numbers of
people at each end:
- innovators
- early adopters
- early majority
- late majority
- laggards
Conclusion: if you build it some people come to it slowly...
Moving from the "early adopters" to the "early majority" is difficult -- it is known
as
bridging the chasm.
If your solution doesn't cross the chasm then it does not
enter the mainstream and will vanish from sight.
Clayton M Christensen wrote a very good book (
"The Innovator's Dilemma: When new Technologies cause Great Firms to Fail",
Harvard Business School Press Boston MA 1997 ISBN 0-87584-585-1 CR9802-0061
). He documented the way that a new technology enters a market
and then comes to dominate it. Paradoxically, initially the old system is
more effective than the new one. Only evolution through usage makes the new
system better. Thus new technology is rarely introduced by companies that
already have a dominant product. The technology comes from a start-up with
nothing to loose.
Some large companies handle this by creating separate parts charged with
coming up with a new product which must then compete with the old
internally. Examples are the Java language developed inside Sun and The PC
developed inside IBM. You may find this branch of the organization called
the "Skunk Works".
Notice that this also means that technological innovations tend to
start with a backward step. It takes several iterations to
get a better system that people are happy to adopt. For this reason you
would be wise to always develop projects as a series of iterations with
time to test ideas with the stakeholders well before the deadline.
(phase): a section of time when a particular part of a method is followed.
(iteration): a cycle through a series of steps that moves something closer to its ideal state.
You will often find that a piece of software's value depends on how many
people are using it -- the more the better. This effect was first noted by
economists with respect to telephones: the value of owning a telephone
depends, critically, on there being people you can talk to using the phone.
Hence the name: Network Effect or Network Externality.
In software, for example, it pays to have a the same word processor
as everybody else. The value of your word processor is higher if lots of
other people also use it.
So, as a rule, a better connected product does better.
An
example is Apple's Newton that did not interface with their Macintosh
series -- and failed. On the other hand, Palm Pilots have always have
HotSynched with PC's -- and are still with us. And of course now Apple
has the iPhone, iPods, and iTunes that connect with PCs and Mac's.
A wise designer will design a new system to communicate well with the old
systems so that people using the new system are not isolated.
This might be called
- Forsters_law::="Only Connect", from a famous novel.
Exception: in the presence of malicious and infective software it can pay to
be different. Diversity slows the spread of viruses.
Never forget that software doesn't have to be developed internally. You can
- Get it free from Open Source projects.
But it may be without support and documentation.
[ open_source_ado.html ]
- You can buy it.
- You can buy it and customize it.
- You can reuse existing software with new software wrapped around it.
- You can outsource development: next door vs off shore. NearSource vs FarSource.
Some organizations require you to prepare a "statement of work" as
part of developing a project proposal.
- statement_of_work::document=following
- It defines: name, manager, client/customer, sponsor.
- It gives dates.
- Staffing: How many people doing what.
- Vision, goal, objectives, phases or iterations.
. . . . . . . . . ( end of section Document -- Statement of Work) <<Contents | End>>
Some organizations require a description of the project in more detail -- a
- project_scope::=following
- Who? What? When?
- General information.
- Problem or Opportunity.
- Objectives.
- Description.
- Benefits to business.
- Deliverables.
- How Long will it take?
. . . . . . . . . ( end of section Document -- Project scope) <<Contents | End>>
- Is this project worth doing -- This is Feasibility.
- Classify constraints: external vs internal, mandatory vs desirable, present vs future.
- Learn the kinds of feasibility
- Operational -- Can it work?
- Technical -- Can we make it?
The key part of technical feasibility is whether we have (or can get) the necessary data that the
project will use.
- Economic -- Will benefits outweigh costs?
If so when does this happen?
What is the Return On Investment?
- Schedule -- Can we do it in time?
- Legal (AMOW)
- Political (AMOW) -- who inside the enterprise will oppose this?
- Ethical (AMOW)
All of these can make a project fail. The most important one varies from
project to project: the critical one is the one that will kill the
project.
(AMOW): I added three kinds of feasibility (Legal, Political, and Ethical) that are not found in most texts.
They are based on my experience of projects.
Commonly you will find several solutions. But a solution
that is good for one criteria is bad for another and vice versa.
For example one solution is cheap to develop but inefficient to operate,
whereas another
solution will be expensive to develop but cheaper to operate. We
talk about this as a Trade-off. In computing for example
we can nearly always speed up a program by giving it more space to
run in. This called a space-time trade-off. In these
cases you can often developed a mixed strategy which combines the alternatives.
Another strategy is to think about the relative importance of the two opposed
benefits and set priorities.
Costs of a system tend to start high and then slowly decline,
and then stay low for a long time, until the system starts to need extra maintenance and ultimately replacement.
We call such a pattern a
Bathtub curve.
Bathtub curves were first observed for failure rates.... but the same curve
fits the costs of a system.
I use them in definitions:
name::type=definition
so that I can extract them into a data file "dictionary" that my
search engine's uses to find things on my web site.
The notation (Bachus Naur Form or BNF) has been used since the 1960's to define
computer languages. In the 1970's it was extended(EBNF)... and through the
1980s onward I experimented with stretching it to the limit --
eXtreme Backus Naur Form (XBNF).
This is checking to see if we can complete the project within the deadline
with the available resources.
You may have to do it yourself! Or at least go into the source
code and use that to answer questions.
The people. And typically today's problem was yesterday's solution!
Many architectures can support a set of functions.
The best architectural choice depend on the
non-functional requirements
like: usability, reliability, security, cost, performance, speed, latency, time
to develop, TCO, ...
- TCO::=Total Cost of Ownership.
- Initial cost is a lot less than the TCO!
- Check the following things about your architecture:
- OPTIONS: choose the best of three architectures.
Or present three to the stakeholders with a range of costs and let
them pick the best one.
- Scalability -- choose an architecture that will work with more data and usage
than you need. Choose one that can expand as the need grows.
- Performance -- details below.
- Training,
- Reliability -- Mean time to failure, ...
- Life time,
- Fit with Legacy Systems,
- Security,
- Maintenance,
- Security,
- System Management and Support
- Help Desk
- Maintenance and Faults,
- Backups,
- Disaster recovery.
- Obselescence -- cost of recylcing and cleaning up.
- Amip Shah & Tom Christian & Chandrakant Patel & Cullen Bash & Ratnesh
Sharma
- Assessing ICT's Environmental impact
- IEEE Computer Magazine V42n7(Jul 2009)pp91-93
- Evaluate the impact of your system over the complete life cycle from
manufacture to recycling or the dump.
- Security.
- Classic Processing options: Online (OLTP) vs Batch, mixed .
- OLTP::="Online Transaction Processing".
- Classic mix: Provide a mostly automatic system for every day work
that gathers performance data for executives and managers to analyze --
OLAP, data ware house, data mining.
- OLAP::="Online Analytical Processing".
- Classic architectures: client-server architectures --
Notice that a mobile platform provides a thin client free but
you can download an app that makes it a fat client.
- Systems Design Completion: Report, sign off, presentation.
There are three things that effect performance: Location, Limits, and Load.
This complicates choosing the best systems architecture.
Study the three sets of notes below.
We met this in
[ a3.html#The Principle of Locality ]
which covered to facts of systems life. First the place where the data is
stored determines the speed at which you can access it. The technology chosen for
each data store will control the possible speed of the processes that use it.
Second, if the data is a long way away, on a network, then the latency will be
larger.
For example see this
[ http://m.wired.com/wiredenterprise/2012/06/cdn/ ]
story from Wired 2012 that describes how some large computer companies --
Google, Netflix, Facebook, etc. -- are moving their processing into the same
buildings as their ISPs. This means that large ammounts of data have less
space to traverse and so arrives quicker... it also reduces the work needed
by the ISP.
Finally, the position of the data inside a device interacts with the sequence
in which a process accesses the data.
So, choosing the right technology for data stores, choosing the right places
on the network, and making processes access data in the optimal sequence all
add up to a winning architecture.
. . . . . . . . . ( end of section Performance depends on the location of the data) <<Contents | End>>
A common mistake is to by a faster CPU when the system is actually slowed down by the disk. It is important to understand what limits a system's performance. Further new technology changes the rules.
Read this article
[ 001304.html ]
(Coding Horror).
. . . . . . . . . ( end of section Performance is Limited by the minimum resource available) <<Contents | End>>
- Performance degrades for high loads. There is a lot of theory
worked out for the behavior of systems under various loads
[Queuing Theory].
A useful rule of thumb predicting performance is that
- (eq1): time = constant /(limit - load),
for some unknown constant and limit.
- Given some low load measurements it is not difficult to estimate what
the limit is. If we rearrange eq1 above as follows
- limit-load = constant/time,
- load = limit - constant/time,
- Now we can see that plotting load vs 1/time we should get a straight
line that touches the x-axes at the limit.
Better we can use statistics to fit the line and make estimates
of the expected value of the limit and 90% confidence intervals for it. To
learn more -- take MATH262!
By how much growth the system can take before it suddenly becomes
more expensive. Typically a small increase in volume or size
is handled easily and inexpensively, but there comes a point
when the added demand makes the system hard to use/unreliable/slow...
Scalable systems respond linearly to growth -- costs go up on
a straight line not a curve.
. . . . . . . . . ( end of section Performance depends on load) <<Contents | End>>
Most effective processes for getting data in the right place etc. are iterative
and based on complex spreadsheet based models. We don't have time to do any of this
in this course, sadly. You may find it described as "Physical Design Control"
in some enterprise methodologies.
. . . . . . . . . ( end of section Designing for Performance) <<Contents | End>>
. . . . . . . . . ( end of section Performance) <<Contents | End>>
Your design should be a good design for the whole life time of the system.
This is an arbitrary time and
if the system works well the life time will be longer than you
would ever believe possible. But in planning: look at how
long it is before a system will be replaced. For planning activities within
a project follow these guidelines
- Precise estimates and plans are only good for a short time. You can
plan a day of work precisely but you will be wrong about six-months.
- Divide a long time period into milestones. Divide these into
"inch" "pebble" pieces. Revise the total plan as you go.
We will cover the classic techniques for detailed planning later.
I call this the most critical quality of the system. Sadly, it varies
from project to project. The prime skill you need to learn is to
spot the most critical quality -- the one property that is most
likely to make the stakeholders reject your system.
For example, if you are producing a piece of software for the mass market
the most critical quality is time to market and just about all the
others (reliability, usability, completeness, ...) have been sacrificed
to beat a rival company to the market.
For NASA however reliability tops the list... but they also have hard
deadlines (launch windows).
For many businesses the most critical non-functional requirement is
maintainability: being able to cheaply make changes in running
software.