Thread: Developers Blog

Hello, everybody! This is the final installment of our blog series "Celebrating the Creative Player" for 2010, and the entire EVE Online Community team would like to thank all of our players for their continued creative expression. EVE is a remarkable game that seems to inspire so many of you, and we continue to be in awe of the talent inherent with the EVE community.

In this edition, CCP Zymurgist and I will showcase some of the more interesting things we've come across in the last month that players have created. We hope you take some time out of your busy holiday schedule of blowing things up to check this wonderful creations out.

EVEfit Project has just introduced Pyfa, or the Python Fitting Assistant. This cross-platform fitting tool allows you to work on fitting your tool with a variety of different modules off line. In the future, Pyfa should have a Web-based tool available for players to play with. It's one in a long line of excellent fitting tools for players, such as the famous EVE Fitting Tool.

Speaking of fits, this site literally had CCP Navigator rolling on the floor laughing. At first, I thought it was because I was in Reykjavik and looked dorky, but it turned out he was really getting a kick out of LOL FITS, a new site showcasing some of the pretty bad fits players come up with in-game. To be honest, I'm kind of shocked none of my horrible combinations are showcased here... yet.

YouTube user cilayin has created an amazing new video showcasing the new character creator tool that will be coming with EVE Online: Incursion 1.1 in January, 2011. This video actually showcases both sexes and the stunning detail the character creator will bring to EVE Online.



Remember, if you create something that you'd like to share with the EVE Online community, we encourage you to create a post in the My EVE channel.

From Zymurgist and I and all of CCP, thank you for being not only passionate EVE Online players, but creative ones as well. May you all have a safe and joyous holiday! Until next year... fly safely!











More...

Contracts Improvements

As a part of the Incursion 1.1 expansion in January we are including quite a lot of improvements to the aging Contracts system. There are a lot of things in this system that could be done better, and over the years we have gotten quite a lot of suggestions from players on how to improve the search part of Contracts.

Backend work

The reason that we are doing this now primarily stems from backend performance issues. Today contract searching is done on the database level and contract management (looking up contracts for example) is done on the location node where the contract was created.

The reasons for doing this were that it provided natural load balancing into the system and it used the database for what it's very good at. However, in real life this is not giving us an advantage because of the fact that by far the majority of contracts are created in the Jita system and the majority of contracts searches are likewise done from the Jita system. So in actuality most of the searches and lookups are being done on a node that's already busy handling the most crowded solar system in EVE because of the massive congregation of people there with the singular purpose of being good little consumers.

This, coupled with the fact that the contracts searches are extremely taxing on the database caused us to take a good hard look at the system. In this push, The backend Contracts system is being completely re-architectured and searching has been moved from the database to the sol node layer onto a dedicated Contracts node. In addition, all contracts live on that specific node (instead of 90% of them living on the Jita node).

The end result is that, with the new system, your location node will have nothing whatsoever to do with contracts, except at the step when items are moved into or out of a contract. This frees up the location node and should make contracts much faster in places like Jita and, even in extremely heavy traffic systems, searching Contracts for an exotic dancer for that special someone should be relatively quick.


This type of work continues on the Character Nodes load balancing work and moves more of the workload off the location node into its own segregated part of the cluster.

Because of this work, we have been able to add a whole host of features that the old architecture didn't support. You know you're doing something right when you can make the cluster happier at the same time as you're adding new shiny features. :-)


Let's delve into the new features in more details below.

Experimental Contracts search

The new Contract search feature is Experimental, which means that it will be in a Beta phase for a few months while we iron out the kinks and make sure it performs well enough. That means that you can continue to use the old contracts search and you can provide us with feedback through the forum thread associated with this blog. Then, a few patches later, we will tear off the Experimental band-aid and let the feature out into the wild. In its experimental state the feature should be completely usable though, and you should feel free to use it as much as you want to when it hits Tranquility.


The new Contracts search and the new Neocom are the first features to get this new 'experimental' treatment. This new way of pushing out features is something we are trying out as a thinner way to deploy things that we want to try out a bit before making the leap to the mainstream. Hopefully this goes over well with the players and gives us good feedback on continued development of the features.


Keep in mind that this is a BETA feature and as such there might be some issues with it, and it might be turned off without warning if it turns out the servers are melting.

You can stop reading and go to the Singularity Test Server right now to play around with this feature but if you're still here let me tell you about some of the features of the new contracts search.

Go to the Contracts start page to enable the feature:



Once the new Public Contracts Search has been enabled you should have access to this tab:



OMG, there is some cool stuff there. Now, on to the list of goodies!

Sorting done on the server

One of the biggest gripes of contractors has been the fact that the entire contract list is sorted by newest-first and you can then apply your own sorting just on each page of 100 entries. This makes searching very difficult since there might be thousands of contracts to choose from and unless you refine your search way down it is a big headache to go through the results in any sane manner.
The new architecture allowed us to change that. Now sorting is done on the server on the entire result set before it is sent down to the client. You are therefore able to sort all the returned contracts by the following sort conditions both up and down:

• Date Created
• Time Left
• Price (Auction and Item Exchange)
• Reward (Courier)
• Collateral (Courier)
• Volume (Courier)

I am sorry to say that we cannot add sorting by distance since that is a user-specific pathfinding search that is simply too expensive for the server. You can still sort each page by distance.
The new contract search will return a maximum of 1000 contracts, or 10 pages worth. These contracts are though properly sorted against the entire pool of contracts so there is no invisible implicit sorting going on.

More information in the list

The old search didn't have a lot of information in the search results. The reason for this was that you could search for multiple unrelated contract types at once and therefore the entries needed to be very general and much too general to be very useful. For example, results for a courier contract search look something like this:



We have revamped this in the new search window and put the focus on searching for one contract type at a time (you will also be able to search for auction and item exchange at the same time, just with less information). This allows us to have the result set type-specific and add relevant columns for each search type that you can then sort by.



As you can see from this example of the courier contracts you can sort the page by drop off location, volume, route length, etc. If you search through auction contracts you get a different set of columns relevant to that contract type.
As you may also notice, the list contains information about the region name if the contract is outside your own region and also shows the security level of the systems in question. The list also tells you at a glance whether your autopilot will allow you to reach the destination.

Location search options

You can search by an exciting array of locations. You can pick your own station, solar system or region as before, but you can also pick your constellation, or another solar system, constellation or region.
This allows you to conduct very versatile searches. Let's say that you found the module that you're interested in somewhere. Now you can search the constellation where the module was sold in to see if you can find something else of interest close by.

Find Related

You now have access to a "Find Related Contracts" menu both in the contracts search results and in the contract menu in the contract details window. This menu allows you to search for contracts from the same issuer, from the same solar system/constellation/region or in the case of courier contracts, to the same solar system/constellation/region.
This is simple but a very powerful productivity tool allowing you for example to quickly gather together all the courier contracts heading into a particular region of space.
For auction and item exchange contracts with a single item you can use this function to find the same item type, group or category.

Courier search options

When searching for courier contracts you can now specify the range or reward, collateral and volume. You can also pick where the contract should be delivered to. Like before, this can be a solar system, constellation or region.

Page filters

Some things we cannot search for on the server, but the client can it quite effectively. Therefore we added page filters which allow you to filter each page of your result set. When these filters are set the number of contracts per page is reduced accordingly.
Some of the things that you can set are:

• Exclude unreachable
• Exclude ignored issuers
• Max jumps
• Max route length (courier only)

Partial Type search

A big gripe for people that are using the system extensively is the fact that when looking for specific types you can only search for a single type at once. The new architecture allowed us to change that.
You can now enter a partial name into the 'types' field and the search is done on all the types that match that partial string. You can also specify a category or group as well as a partial type name to further refine your search.
By the way. You can drag items into this field to search for them and you can drag people from your people and places or chat into the 'issuers' field which can be very convenient (not on Sisi yet).

Filtering out the scammers

You can exclude contracts with multiple item types and the Want To Buy and Want To Trade contracts in your searches as well as exclude systems and stations that are unreachable. This should make your browsing a bit safer and less annoying.

UI is still under construction

The User Interface to the new contracts search is undergoing revision as we speak. You should expect this to change and improve over the next few weeks as CCP Karkur works her magic. If you have usability suggestions, let us know!

Uh, oh. It's only for public contracts

The new contracts search does not search through private contracts, only the public ones.
The reason for this is that there aren't that many contracts assigned to each alliance, corporation or character so all the features of the new search aren't as important for these private contracts. Since the searchable contracts cache does cost quite a bit of server resources and the number of private contracts overall is pretty great we have opted for using the old contracts search for these types of contracts.
If you would like to extend the new contracts search to cover the private contracts as well please let us know since we haven't seen the reason to do so.

A fat pile of features

In addition to the new contracts search we have added a bunch of new features and usability tweaks to contracts, most of which were requested from players on the forums and by the CSM. Thank you for all the great ideas, I think we've got most of them in by now.

Copy Contract

You can now "copy" a contract that you have created from the contract details context menu. This will open up the "Create Contract" wizard with parameters from the copied contract prefilled in.
This is especially useful if you want to create several identical contracts and should reduce the clickfest by an order of magnitude or so.

Insured ships

Insured ships that are put into a Courier contract will no longer have their insurance voided. Contracting insured ships should be more consistent.

Notifications

When you log in you are now notified if there are any contracts assigned to you and you can easily jump into the contract in question. Contracts that require attention are similarly highlighted so you should have a clear overview of your contracts status when you log in.
This uses the same notification mechanism as skill training complete.

Item Volume

A new "Volume" column has been added in the "Select Items" step of the Create Contract Wizard. Also, when creating a Courier contract the volume of selected items is summed up when they are selected. This should make Courier contracts much easier to create.

Damaged Items

Damaged items can now be put into all contracts. If an item is damaged it is displayed in the Contract Details window. This applies to all items and all contract types. No more hassle with contracting ships containing T2 crystals!

Solar System Information

We are now showing a lot more information about the solar systems a contract is in to give you a better picture of whether its safe to go there.
Contract details will now show the security level of the systems in question and will also show if any system in the route to the system is of another security level than the current system. You will also be able to see if a station is player owned and therefore potentially unreachable or if the system is unreachable with your current autopilot settings.

Find Contract for plastic wraps

Added a "Find Contracts" context menu option for plastic wrap containers to allow you to bring up the contract which the container belongs to.

Find in Contracts

You can now right click on any item or item type and click 'Find in contracts' to search for the type in the public contracts (providing you have the new contracts search enabled)

Loaded charges

When you contract a ship with loaded charges the charges are now removed into the ship's cargohold rather than into the hangar. I'm sure this will make a lot of people happy.

Fitted Modules

Contracts with ships in them show a 'fitted' tag on each item in the ship if it is fitted on the ship rather than being in a cargo hold. This is especially useful for seeing if a rig is fitted or not, since they cannot be removed from the ship.

Other tweaks

• Added a "show info" context menu option for search results containing one item.
• Create contract wizard has been simplified a bit.
• Item Exchange Contracts now have a different icon. We repurposed the "freeform" contract icon for this type of contract.
• If you accept any type of contract you should be correctly notified if the station(s) are player-owned and not reachable.
• Metalevels of items should now be correctly displayed everywhere in contracts.
• You can now preview items from the contracts details window.
• Contracts should now open up more quickly than before. No more 'shuffling through the pile'.
• A bunch of other small fixes have been added here and there.

On the horizon

We have a few contracts feature requests on our list such as saving contract search conditions (quicksearch), bookmarking contracts, contracts API and more usability tweaks. We will hopefully be able to add some more features over the next few months.

If you have suggestions for improvements by all means let us know on the feedback thread. We would love to hear from you.

Jon Bjarnason
Technical Director
EVE Online, CCP Games









More...

Comparing this ancient screenshot of the EVE client to what we are currently used to stare at day in and day out reveals many interesting things. During those 7 years that have passed the ship HUD has had a substantial facelift and the threats window has grown up to become an overview. One of the few things that hasn‘t changed much at all, if at all, is the NeoCom. During those early days of New Eden the NeoCom did its job just fine, but as EVE has grown with every expansion delivered, it has become more and more obvious that a more elegant solution is needed to allow for easier access to the tools and features we use on a daily basis.



So we did this:



The two major aims of the new design are to increase customizability so pilots can configure the NeoCom bar to fit their daily needs as well as making the ever growing number of features in EVE more easily accessible.

The EVE menu

This is where all the features in EVE can be accessed at all times. It‘s not customizable; that‘s what the NeoCom itself is for. The introduction of the new group structure means that we don‘t have to hide new exciting features under a tab in a semi-related window because there isn‘t more space in the NeoCom.



Customizability

The customization of the old NeoCom was very limited to say the least, but no more. You‘ll now be able to decide exactly what buttons are displayed in the NeoCom and reorganizing the buttons is as easy as dragging them around. Removing a button from the NeoCom is a right-click affair and re-claiming that same button is as easy as dragging it from the EVE menu into the NeoCom bar. It is even possible to create groups into which buttons, and other groups, can be dragged.



The size of the new NeoCom is also configurable by dragging the top border, ranging anywhere from this:



... to this:





Other features

• A skill training progress bar that also allows for one click access to the skill training queue
• The old "minimized window buttons" functionality is replaced by displaying icons of all open windows in the NeoCom. Windows of same type (multiple "show info" windows for example) are grouped together into a single button.
• Buttons with special functionality:

o Chat: Groups all currently open conversations and has a button for directly accessing the "channels" window.

o Browser: Lists all bookmarked sites, making them easily accessible.



It‘s still in BETA



Releasing new features as BETA before fully integrating them into the game is a new concept we at CCP are trying out and the new NeoCom will be one of the first features to get such royal treatment. The idea here is to make the introduction of new features a smoother experience, both for developers and players. To enable a BETA feature, one must go to the "General settings" tab of system menu (ESC) and press the relevant button in the bottom left corner. Should this iteration of the new feature not meet your expectations, you can simply turn it off for the time being and tell us why you did so on the forums. Even though SISI has proven a great play testing tool, we feel confident that trying out new features on TQ where you are actually playing the game will provide better feedback than we‘ve been able to get in the past.



We already have plenty of ideas for iterations beyond the BETA, such as allowing alignment to any side of the screen, customizable names and colors of groups and adding more visual polish but we want more! Is there anything blatantly missing from the current design? Are there windows you would like to be directly accessible through the EVE menu? Do you have an idea for a special NeoCom button that would make your life easier? If so, please don't hesitate to tell us on the forums (and we might listen).



The new NeoCom BETA will be hitting TQ as part of the main Incursions release, January 18th 2011, and SISI before Christmas.



CCP Optimal











More...

It's been almost one and a half years now since we pushed ahead with operation Unholy Rage in our campaign against RMT. Much ground has been gained during this period, most notably in the field of spam control, in limiting the damage from account hacking and in the promotion of PLEX as an alternative for those looking for more ISK. Hundreds of accounts are banned every week for macro use and other RMT related activities, and we constantly explore ways to help us adapt to the ever changing methods of the multi-headed monster we're up against.

While we have put much of our focus on the RMT related activities we do realize that botting is not exclusive to RMT in EVE and we do investigate reports and take action against players proven guilty of using macros for mining, ratting, mission running, market activities and whatnot. There are some fairly refined macros out there and it is not always that easy or straightforward to establish beyond doubt that someone is using a macro or not. This is something we are working on all the time, and we do intend to uphold our rules and policies in regards to macro use in general, not only the RMT part of it.

We appreciate the reports you guys keep sending us about suspected bots and so on and would like to use this opportunity to thank you for that and encourage you to keep those reports coming. Also, use of the "Report ISK spammer" feature has been very helpful in the battle against RMT - also something we'd like to commend you guys for.

We have by no means stamped out the whole of RMT in EVE Online. This is an ongoing struggle and the results depend on what we do - but they are also dictated by the conduct of you, the players.

ISK is sold because people buy it. Players do business with the RMT element and that is why accounts are hacked, asteroid belts are sucked up by bots and it is why we find ourselves in our current situation.

We recognize the need of those who crave more ISK but do not have the time or in-game means to gain it. This is why we have created an alternative way for players to get the ISK they require by safe, perfectly legal and acceptable methods.

PLEX is available on our website in handy packages of various sizes and once redeemed it can be traded for ISK on a fluid player driven in-game market.

Since PLEX was made available on our website we have seen a steady increase in activity and regard this as a healthy progress in the fight against the abhorrent and lamentable RMT affliction.

Education and heightening of awareness constitutes a major requirement in our battle plan. Many of the ISK buyers we process claim ignorance of the rules and we must make sure that information is available and prominent. The risks involved in dealing with the RMT element need to be clear to new players and old - wallet in the negative after bought ISK is removed by GMs and a warning on the account - something we all wish to avoid. Doing business with the RMT element also encourages all kinds of undesirable and often outright illegal activity such as account hacking, macro use and credit card fraud.

We prefer that the way of the PLEX be known to and embraced by the EVE community at large for it brings great advantages - to the buyers, to the free-to-players and to this company to aid maintenance and development of EVE Online for an enhanced gaming experience for everyone.

We are committed to this mission. Our tools are continuously under revision and development and we have people on the job who possess ample talent, constant resolution and the will to fight - and win!

If things seem a little bleak at times - have heart, have faith...

We are still in the fight.











More...

Previously...

In the first part of this blog, I talked about what Destiny is, and showed how a tool called Telemetry is helping us visualise where we should focus our efforts to improve server performance. I ended with a highlight of a particular problem - missile spamming Drakes causing excessive Destiny load.

This time, I'm going to show how I changed the situation for the better.

I'm going to begin at the end. Or, more specifically, I'm going to show the results. Figure 1 shows the overall allocation of CPU time during 100 seconds of missile spam. (For the specific setup used in the test, see the first part of this blog). The left column shows where the time was spent in the old code, as it has been on TQ for a long time. The right column shows the new improved version, released with Incursion.



Figure 1: Total CPU load over 100 seconds - before and after the change

As you can see, the Destiny load has been substantially reduced. Inventory and damage has now become the single most expensive area - we have already got some promising improvements in the development pipeline for this, but those are for another day.

So, how did this happen?

Warning: Pseudo-code ahead!

One of the first things that Destiny does each tick is build lists of added and deleted balls for each observer. For each added ball, we also have to get extra data, such as corp ID, security status, visual effects (such as Sensor Boosters) as well as the actual dynamic state (position, velocity and so on).

Vastly simplified, the algorithm for doing this looked something like:

1: for every observer: # (ie a user's ship)2: for every ball in the observer's bubble:3: if ball was added/deleted to the bubble in the last tick:4: gather up all the data required to tell the observer's client about this ballIf you think about this for a while, it might become obvious why this scales so badly - O(n*m) is bad when n or m gets large. When you consider that step 4 always produces the same data for a given ball, regardless of who the observer is, you realise this is also incredibly wasteful.

The first optimisation was to add a cache to step 4. For reasons I don't have space to explain, this actually benefits fleet warps more than stationary engagements. It also doesn't get rid of that awful nested loop.

The next optimisation was to refactor the algorithm. At this point, I want to detour slightly to address another common concern: Why do you use Python? Python is slow. If you ported everything to C++, there would be no more lag. Well in this case, I have to disagree with those statements. The function in question was responsible for over 15% of the entire CPU load in some mass-test I have monitored. It was also written entirely in C++. The fact is that a sub-optimal algorithm can be significantly more costly than any overhead due to language choice. For low numbers of observers/balls, the algorithm described above is perfectly reasonable. However as we push bigger numbers at it, it is going to scale in a very poor manner.

The new-and-improved version of the function now looks something like this (again, vastly simplified):

1: for every bubble:2: create a list to track adds/deletes:3: for every ball in the bubble:4: if ball was added/deleted to bubble this frame:5: gather up all the data required to tell a client about this ball, and store in the list6: for every observer: # (ie a user's ship)7: store an index into the list created in step 2, based on observer's current bubbleThis now scales much better as the number of balls and observers rises - it is more like O(n+m).

The other area of Destiny load I identified previously was the overhead of serialising the updates for each client. Solving this was one of those epiphany-in-the-shower moments that could only happen after I'd made the changes above: Most clients in the same bubble will received a serialised update containing the same information. Why serialise that information for every single client, when I have already determined the part they all have in common? After we've completed step 5, we can pre-serialise the update once, and simply pass that same blob of data to every single client in a given bubble. Previously we did one serialise operation per observer. Now we do one serialise operation per bubble, followed by one memcpy per observer. (This is vastly simplified - there are still edge cases that this doesn't cover, but those are in the minority)

The valley of many Awesomes

The improvements are written in such a way that I can toggle them on and off whilst the system is running live. This graph of CPU load across time shows the effect of turning the optimisation on and off:



Figure 2: CPU load during missile spam, whilst toggling the optimisations on and off

Missile spam starts at 15:21, and they run out of ammo at 15:27.

Initially the optimisation is switched off. At 15:23, I switch it on (CPU drops dramatically). At 15:25, I switch it back off (CPU rises back up)

For comparison, here are some Telemetry traces of Destiny ticks with the optimisation off and on:



Figure 3: Comparison of two Destiny ticks, before and after the optimisation (horizontal scale is the same)

The upper trace was taken just before 15:23, and the lower one just after 15:23.

Deployment

These changes were running on Singularity during October/November. If you were present at some of the mass-tests, you may recall how we organised multiple fleet-warps, gate-camps and POS bashes in order to exercise this new code. This proved very useful to validate that my experimental results were still applicable in a real situation with real users.

Client support for the new pre-serialised updates was deployed with Incursion 1.0.0 on November 30th. On December 1st I activated the new Destiny code on a small number of Tranquillity nodes, and closely monitored the affected systems. Once we were happy everything was running correctly, the changed was activated for all systems during downtime on December 2nd.

Hopefully you've found this behind-the-scenes look at Destiny interesting. If so, let us know and we might do more. Feel free to ask questions in the discussion thread - I'll keep an eye on it and try to answer what I can.











More...

Listen Up

Hi. I'm CCP Masterplan, one of the members of Team Gridlock. Our job in Gridlock is to tackle the issues of degraded performance under excessive load - in particular making EVE's signature massive fleet-fights more fun.

Whilst CCP Veritas has been having lots of fun switching modules on and off, I've been looking at another area of server load: Things in space. Specifically, adding and removing them, moving them around, and how we tell your clients all about them.

You might have seen mention of something called Destiny, and how it is something to do with physics. In this blog, I'm going to talk a bit about what Destiny does, and how I've been profiling it. In the second part, I'll show how one improvement in particular applies to my current nemesis: missile-spamming Drakes.

The test

The first step in improving performance under heavy load is to be able to reproduce a situation where load occurs. Previously this was done during mass tests. However thanks to the wonderful thin-clients, we now have much simpler ways to recreate different aspects of fleet fights on-demand.

We know that Drakes are currently a popular ship, and we know that they can fire lots of missiles. All the following results were obtained using this same scenario:

• 200 thin clients flying Drakes, each with three fully loaded heavy missile launchers (representing three grouped launchers)
• 100 thin clients flying other ships. These were present to add ‘observer load' - representing the other people involved in a fleet fight. This is important, as the cost of a missile is somewhat proportional to how many observers can see it.
• 1 control tower with a ‘death star' set-up. The tower was configured NOT to aggress attackers.

Each Drake was instructed to lock the control tower. They were then instructed to activate their launchers, firing until their loaded ammo was depleted - this gave around 5-6 minutes of steady-state load (AKA missile-spam).

The numbers of ships was chosen (by trial-and-error) to push the CPU load of the node to around 95% during the steady-state missile spam. This is an important level, as it represents the edge of performance where lag starts to occur (module activations start to become delayed, movement commands stop being responsive, jumping becomes difficult, and so on.) These absolute numbers can't be directly compared to TQ due to hardware differences, but the relative proportions can.

During the steady-state load, we can then use profiling tools to analyse where we are spending CPU time. We can measure any improvements against this baseline of 95% load. Note that this test is entirely aimed at tackling Destiny load in the case of a stationary (non-warping) fleet battle. As such, it doesn't cover other bottle-neck areas such as jumping (though other work has focused on this). Turret-based damage was also not considered, since this has been observed to cause lower load relative to missiles. However this is an on-going effort - as the load of one aspect is reduced others will become proportionally larger and get their share of GridLock's Special-LoveTM.

I'm your density. I mean... your destiny

Before I show you the results of the profiling, it would help if I explained a bit more about Destiny. We often talk about Destiny as being the physics engine of EVE. Whilst that is an important part of what it does, as an overall system it is responsible for much more as well.

First I need to define a couple of terms, so everyone knows what I'm talking about:

• Ballpark: Manages the space occupied by a single solar-system. A ballpark is a collection of bubbles.
• Bubble: A small volume of space within which entities can physically interact. (Players have come to refer to this as a ‘grid') Not to be confused with Warp Disruption Bubbles, which this report is NOT about. Bubbles shrink and grow dynamically, and can never overlap.
• Ball: An entity (ship, drone, missile, asteroid, NPC, wormhole...) in space. As far as Destiny is concerned, all objects in space are represented by one or more balls. As the name suggests, the simulation treats them as spheres.
• Tick: Destiny operates in discrete ticks. Each ballpark runs at a rate of one tick per second.

That finely-shaped engine of the universe

Each tick is comprised of three main steps:

1) Pre-tick

• Update the bubble-ball relationships
• Apply any client actions (Warp, Orbit...) that came in since the last tick
• Apply any server events (Explode, spawn missile...) that happened since the last tick
• Build client updates, so they know what had happened
• Send updates to clients

2) Evolve

• Move balls according to their current action
• Resolve collisions between balls
• Dispatch collision callbacks

3) Post-tick

• Handle new clients (due to jump-ins, undocks etc)
• Cleanup

Once per second, we go through these steps. Whatever time is left over is then used to process everything else that needs to happen. It is important to note that the Destiny tick is entirely non-blocking - we can't do any asynchronous operations such as database queries or anything that might cause execution to yield. Most of the collision callbacks, for example, simply schedule a task to be run later. In this task, we can than perform database operations we need, such as creating wrecks, and moving characters to the clone bay.

This is why, in loaded situations, there is sometimes a delay between a missile hitting a target and that target actually exploding. Sometimes this is even long enough for the target to warp out and think he is safe. In fact, he is already dead - he just doesn't know it yet!

The Evolve step is the most complex part, at least in terms of hardcore maths. This is where all the physics of spaceflight happens. (At least EVE's version of spaceflight physics) It has long been assumed that this was also where the majority of CPU time is spent - and so where the biggest savings could be made. Suggestions such as ‘GPUs are good at math - why not implement this on a GPU?' have been raised. However, as we'll see shortly, the Evolve step is responsible for such a small portion of the overall load that such efforts would probably be counter-productive once the additional communication overheads are taken into account.

Our favourite new toy

I'd now like to introduce the latest weapon in our anti-lag arsenal: a profiling tool called Telemetry. This is a tool for visualising real-time application performance, developed by RAD Game Tools.

We started evaluating this over the summer, and have been finding more and more uses for it. Essentially what Telemetry does is build a timeline of events as the code execution moves in and out of named regions, and then provide a visual way to analyse this execution data. Combined with the thin-clients, we now have a really good way both to generate load, and to visualise it. This makes us in Gridlock very happy.

Figure 1 shows how Telemetry displays a particular Destiny tick. Time is along the horizontal axis (increasing to the right). The vertical axis (increasing downwards) shows how execution enters and exits different timer sections. This looks much like the evolution traditional call-stack over time, just using named timer sections instead of function calls. When a new timer section is entered, a new block is added below the previous one. When a timer section is exited, the corresponding block is removed.

(Programmers are odd creatures. Not only do they start counting from 0 instead of 1, they tend to view structures such as stacks growing down, rather than up. It is just the way we do things...)



Figure 1: A typical Destiny tick during missile spam, with the three main steps highlighted

Looking at Figure 1, we can see that less than 10% of the overall time in Destiny is spent in Evolve. In fact, the majority of Evolve is spent in the collision callbacks - in this case scheduling missiles for explosion. That is why I mentioned above there is little to be gained right now in making the actual simulation code run any faster.

The majority of the time is spent in the Pre-tick step. In particular, two operations make up the bulk of the time:

1) For the list of balls which were added/deleted since the last tick, figure out which clients need to be informed. When a missile is fired a ball must be added to space - when a missile explodes a ball is removed from space. Drakes are good at making this happen a lot!

2) Sending an update to each client identified in the step above. Further investigation showed that this is mainly expensive due to the time required to serialise each update message. (Serialising means converting a data structure in memory into a stream of bytes that can be sent over a network)

If we can find a way to make each of these steps more efficient, then overall Destiny load will be reduced. Well, as it turns out, there is indeed a way. But to find out how it works, you'll have to wait for the second half of this blog...











More...

It's that time of year again, when people get into the spirit of giving, even the people 23,000 years into the future. CONCORD has joined in the seasonal festivities by packing its hangars with a very special gift for the capsuleers of New Eden. December 14th, the pilots of EVE Online will receive the latest ship to roll off of the CONCORD production line - the Echelon.

(Click to Enlarge)

Constructed in secrecy, the Echelon is a minor masterpiece of engineering, integrating otherwise incomprehensible Sansha technology into a frigate-class capsuleers vessel. The advanced cryptographic processors on board provide superior hacking capabilities for use on most known password-secured devices. However, the true beauty of the Echelon is in the adaptive synchronization of capsuleers and Sansha codebreaking components, creating a system predicted to give capsuleers an edge over even the strongest of the Nation's encryption locks.



Beginning Dec 14th 2010, you will be able to redeem the new holiday ship on your account login screen. You can then assign it to a particular character and CONCORD will deliver it to your hangar bay. You will be able to redeem the holiday ship once per active (not trial) account and will no longer be available after January 3rd, 2011, so make sure you log in to receive yours.

Here‘s how to redeem it:

• Log into the EVE Online client.
• At the character selection screen there will be a Redeem Items button in the lower right corner. Select the character you wish to receive the Echelon and press the Redeem Items button.

• Check the box next to the Echelon and the Purloined Sansha Codebreaker in the confirmation window and press the Redeem Selected Items button.
• You will receive a confirmation window that informs you which character is receiving the Echelon and in which station it will be deposited. They are always deposited at the character's current location.
• Click Yes to accept the confirmation window if you are happy with your decision
• Log in to that character and jump in your new ship!

(Click to Enlarge)

Season's Greetings and Happy Holidays from everyone at CCP!



- CCP Loktofeit



Update: NPC Station and Item Redemption limitations removed from blog as these limitations have been lifted in previous patches.





More...

So this was how it started off, somewhat dramatically, back somewhere in the murky pre-recession depths of 2006:

"I don't know what you saw," she says, "because it's different for everyone. I see snowflakes, gently drifting about inside the boxes. Other people see colors, and others can only tell by feel. Your mind picks whatever you can handle.

"I want you to leave your life," she says. "Come with us. Help put things right. There aren't many people who can do what we're asking you to do."

He's still not sure. Then again, he thinks, what does he have to lose?





And this is how it's going to end, right around Christmas 2010:

The box contained another box, this one made of marble and decorated with impossibly ornate carvings. Skar looked at them for a few moments and felt something in his mind begin to drain away, but the gale of the wind and the patter of the rain brought him back to normal. The marble box also had a faultline in the center but was not sealed, and Skar felt his eyes drawn to one of its corners, where a brownish piece of scroll poked out. A small, tattered piece of the Book of Emptiness, poking its edges into this world.

Skar walked out swiftly, marched a few steps behind the tent, vomited quietly, and walked back into the tent. Akran didn't seem to notice.

That one, I should add, is a story about faith, about the need for faith and the need for some semblance of stability in daily life. The vomiting bit, I assure you, is artistically justified.

***

The EVE Chronicles are going on hiatus for a bit, after which we'll be publishing them less regularly, but more strategically. I want to tell you a bit about how we got to where we are and where we're going from here.

I've been CCP's primary chronicle writer for four years now. I started off as an outsourced GM (as they all were) back in 2003, working in the trenches for a bit over three years while finishing up a Comp. Sci. degree. For fun and relaxation, I wrote stories about EVE, and because I am the kind of well-adjusted, genial person whose hobbies are entirely legal and don't at all require rolls of plastic tarp and duct tape, I naturally wrote stories about people being tortured to death in various inventive ways. I shared those stories with some people I knew, and eventually, this being CCP, I was emailed not a restraining order but a job offer.

It's been amazing. It truly has. The chronicles already existed, nearly a hundred of them, but they'd been published piecemeal over a period of several years and had been written in good part to flesh out a world not yet fully alive and breathing. I wanted to focus more on stories and narrative, to cast a light on the people you didn't see, or only just saw out the corner of your eye. Spaceship crews. Station inhabitants, anything from agents and mechanics to drunkards, artists, and other no-good wastrels. Colony workers, doing their best in terrible conditions. I wanted not to tell us our own stories of capsuleer life in EVE, nor compete with them - how could I? - but to fill in the cracks, make it feel like the world we were playing in also existed outside our own field of vision.

So for the next four years, that's what we went for, in chronicles published every two weeks without fail. I've now published over eighty of my own stories and a good two dozen of other people's. I also wrote two EVE plays, performed by our own amazing thespian devs at the last two FanFests: "Speakeasy", about what agents do in their meager time off and why you wouldn't necessarily want their job, and "The Silver Twin", which is about lies and criminals, dangerous journeys into deep space, and why you don't ever want to fall asleep at a drug-fuelled Gallentean party that's also being attended by bodymodders.

And at some point I wrote an EVE novel, too. It's good. You should totally buy it.

It hasn't just been me, either. Writing a novel takes a lot of time and effort and caffeine, and while I was working on it I wrangled some of our other Content people into writing chronicles of their own. All of them are amazing writers in their own right and I wish we'd had the opportunity to spotlight their talents more. Only recently, Headfirst came up with "Ante" and "Xenocracy", which respectively showed just as much talent for sneaky backstabbing and outright evil as any CCPer is expected to exhibit; Ginger continued his big burly Minmatar obsession with "Valklears" and "Tattoos"; Gnauton and Greyscale covered the clandestine, paranoiac worlds of the Caldari and Amarr with "Chasing Shadows" and "Signs of Faith"; Big Dumb Object had a spaceship crew do a Running Man in "The Rite" (and I hate him for it because I wanted that idea); Jasonitas did horrible things to people in "Lost Stars" and then managed to outdo himself in "Innocent Faces" by bringing in a clown; and Dropbear hit the damn thing out of the park with "Jita 4-4", one of the best-received stories we've had in ages. These guys outdid themselves time and again, and I was incredibly happy to have them join in at last. Except for Dropbear, whom I intend to murder.

What has been lacking, however, is what I'll call meshing. There are the chronicles, and there are missions, and item descriptions and all the rest, and then there are events - Sansha's been up to some dark little tricks and Incursion now looms large - and behind all of this is a world that's very much alive and breathing, and whose various corners really could do with some inspection.

Pop quiz: What's the Gallentean religion like? On average, how many people live on stations, and how are those societies segmented? How do blueprints really work? There are hints here and there, but none of these things have ever been conclusively laid out. Considering the wealth of EVE fiction we possess, there really is a myriad of worldbuilding opportunities. We've wanted to take advantage of this for a while but always ran into the same old familiar problem: Our current way of doing things didn't easily allow for new types of content - we didn't have a way to integrate it with preexisting work, nor a way to satisfactorily deliver it to the players - and besides, we were all way too busy on our current projects to fit in something as sizable as this.

At the same time, it's obvious that while the chronicles enjoy their fair share of readers, they are probably the least-integrated part of the content we deliver on a regular basis. Note that this does not mean they need to be stuffed with more background information - I've been a proponent of "story over infodump" for years, and I'll happily argue the point over any number of beers at Fanfest - but it does mean that when you read something we've written, it should be much more related to other pieces of content.

If you read a chronicle about some person working in a hisec space station, you should be able to meet him at some point in Incarna. If he comes from a planet nearby, it should be a habitable one whose fictional infrastructure and institutions really could have created someone like him, and which are accessible for you to read about when you're in the vicinity. If an agent then hands you a mission that references that planet, it should also reference that same infrastructure, and possibly even the person you read about in a chronicle. And if you want to find out more about any of these, you should be able to do so right away, in depth, and without any necessary foreknowledge. The lore should be deep enough to be interesting to everyone, but also accessible enough - both in how it's organized and how we present it - that anyone, new and old, can find his or her way around it without first having to memorize years' worth of details.

This is what we've wanted for a while, and under TorfiFran's aegis as Creative Director, this is what we're going to do. The first steps involve taking the lore as it exists today and organizing it in such a manner that it'll be readable and comprehensive, fleshing it out where needed (which will be a fairly sizable undertaking), finding the best ways to deliver it to you in-game, and keeping it updated. We're still hashing out the specifics, so I can't tell you exactly how it's going to play out, but thus far we're planning to add small snippets of in-game texts to various parts of New Eden, add links at strategic places to those texts, and have those links bring you over - through the in-game browser - to our very own EVElopedia. Where, if we get it right, something like this will happen:

[IMG]file:///C:/xkcdwiki.jpg[/IMG]

(XKCD)

So while everyone else is doing their thing - Jasonitas has moved to DUST, Headfirst's over in Incarna, Dropbear is plowing through Incursion material, and Gnauton just finished hammering a positively gargantuan piece of backstory into shape - I'll be working on this from now on.

We want to give you what TorfiFrans calls "The Wikipedia Experience," which he describes as a "captivating immersion in a vast, thoroughly interconnected web of information, lore, and trivia that reaches out into space and far beyond," and which I think of as "having you sit by your computer, launching EVE, clicking some in-game link and start to read, only to finally drift back to coherent function hours and hours later, still docked, the fleet battle long since over, the sun risen hours ago, everyone having left the house for work and school except for you, unshaven and crazy-eyed, because you started out reading about some planet in some solar system and suddenly now you're on some page that's describing in excruciating detail what boosters do your innards after you've been using them for three weeks, and you've got another tab open about commercial services on Gallentean stations and the kinds of freeze-dried things they'll send to you if you're not on an offenders' list, and another tab covering religious rites and occult practices with the Caldari Deteis, and half a dozen tabs waiting after that AND oh god IT NEVER SEEMS TO END."

This is a big project and it's going to take us some time to implement. It's effectively split in two: sorting out the out-of-game content for the wiki, and implementing effective bridges to that content from within the game. The former task involves not only extracting tons and tons of data and factoids from our previously published material, but also organizing it for maximum readability and, later on, expanding it with new content. A single Content person could happily do this stuff for years. Aside from dev support we've made the decision to take Mercury (our volunteer fiction corp) off roleplayed news for the meantime, and they are now supporting this effort as well.

The latter task involves a lot of small-scale logistical challenges and a wealth of content unto itself, and while I won't tell you right now how exactly we're planning to implement it, it does involve taking those parts of the in-game UI where you might be expected to come into contact with backstory, and rigging them up to support quite a bit more of it. Our plans for connecting this to actual gameplay, both in EVE and Incarna, are much longer-term, so you'll hear about them some other day.



For now, thanks for reading, and tons of thanks to those of you who've been regular readers over the past few years. Remember, the Chronicles are still available on our web site, and whenever anything new comes out, now or later, we'll keep you updated through newsposts or on my dev twitter at @cloisterphobe. We'll publish the last one (not the last one forever, but the last one for a little while) just after Xmas, and at some point I plan to release a compendium in various e-readable formats.









More...

We figured that with the attention our character creation system has been getting lately, that it was time to write a devblog about it. There´s lots to cover, so let´s get to it!

The Carbon Character System was made to enable us to create characters for several projects, including Eve. It started off as a design document containing various wishlist features, ranging in complexity from the simple to the impossible. I believe most of the items on the list ended up being possible, with the possible exception of essentially being able to perform a striptease with arbitrary clothing. Hmm, do they even do that in CGI movies these days? Do I really want to know?

Anyway, the system has been seen in public before, highlighing cloth simulation and recently on the Singularity test server, as seen here and here. So what‘s it all about? Is there anything new here?

For this blog, which I hope is the first of several, we‘re going to take a closer look at a couple of interesting features of the system you‘ll have in your hands soon. There are lots of interesting (to me, anyway) things about the system to explore, but I‘ll start with what makes the faces tick - the specs of the faces and the face sculpting feature. My vantage point is right in the middle between programming and art - I do character tech art.

Example Eve Character.

Making Faces

Making 3D models look as good as possible can be accomplished in a number of ways, but the easiest is to simply use massive poly counts and huge textures. Getting the models created by awesome artists helps too. The Carbon Character System scales up and down easily, and if you take the time to create all the level-of-detail meshes you want, it can go from simple stick figure to massively heavy character. For character creation, we decided to do everything we could to ensure you see exactly what our artists wanted you to see, resulting in these specs:

• We use a separate base mesh for each bloodline. There are 24 of them total.
• Each head is about 4500 vertices at the highest detail level. The tearduct is a separate mesh...
• All the faces are perfectly compatible with each other - same vertex counts, models, etc.
• To allow sculpting to happen, each face has 92 morph targets, that is sets of different shapes that can be applied to the face to , for instance, lengthen the nose. In addition, the body uses 20 more, some of which are in the face as well.
• At the very highest settings, (not the default ones!) your character uses a set of textures that are 4096x2048 in size - uncompressed.

The models contain a lot of detail, and it‘s all available in-game. Note: There are no shadows in this shot to show everything clearly.

Those textures use up a massive amount of video memory, of course. But since we‘re not bounded by console-size memories and have the textures available at this size, and since they compress really well using lossless compression, we decided to go for it, and allow those with the hardware to enjoy it fully. And we‘re not going anywhere - in a few years, this really won‘t seem that much, so no harm in being prepared for the future.

Our angry trailer guy is actually using game texture resolutions.

Touching Faces

A major feature of the character system is the ability to "Sculpt" the face - pick at it, and have it react to your prodding and pulling in a way that makes sense. This, of course, needs to work on any of the faces, all of which have slightly different skeletons. This is how it came about, and is an example of how the Carbon framework can make seemingly complex systems pretty easy to make.

A peek at some of our raw blendshape data. Each row represents a zone you can "touch".

• First, I needed some morphs with various areas moving in response to picking at the face in front and side view. At first, I just moved some vertices around very roughly to create the first version of the 92 shapes mentioned above.
• Then I needed a picking mechanism, something to allow me to click the face, and have it react. It turns out that Eve ships can tell me where they've been clicked, with UV coordinates. That's very handy, since they're in the same rendering framework and the method could easily be "borrowed".
• Mouse gestures needed to mean something, and not just left-right or up-down. So I prototyped a system in Maya that allowed arbitrary field of 2D triangles to contain value combination that could be interpolated. In theory, it can interpret gestures of any kind for any purpose, but it turns out that keeping the fields simple works the best for sculpting - in effect they ended up pretty similar in behavior to the old-style face shape fields. The Maya prototype code easily ported over to the engine - a bonus of CCP using Python for everything.
• The player needed to see what's going on, something to indicate what area is being moved. So a programmer in the character team copied the character to a "ghost" scene, and made it render the character in a way similar to ship shield hardeners.
• Once this was all working, it wasn't hard to use the same system to pose the character, and here, the fields could be a bit more elaborate. They just controlled animation parameters instead of fading morphs in and out.

Example triangle field. Each vertex can contain any combination of A=x values. The actual fields in the nose ended up simpler than the prototype field on the left.

It took a couple of weeks of prototyping to make it work with placeholder data without feedback. Then a process of gradual refinement and fitting it into the rest of the system took place. For example, if you drag the bridge of the nose, what do you do with glasses? Shouldn't they move too? Yes, the solution is to make glasses inherit the morphs of the faces without distorting them. Not trivial to do, but nothing our MacGyver-esque tech artists couldn't handle.

Future Topics

There are all kinds of other items to discuss in the future. Here are some examples of things likely to crop up:

• Texture compositing - how it's done, and what you can do with it
• Morph shapes everywhere - not just used in the face
• Normals - why they're evil and will hurt you if you let them
• Cloth Sim - blowing' in the wind (or at least the in-station AC draft)
• Grievous Bodily Injury - how injuries could be authored just like clothes
• Giga-what? The terrifying life of CCP artists and their technological tormentors

CCP Ph00ze

More...

Hello, creative Capsuleers! It is time for another edition of the EVE Online Community team's monthly blog celebrating the creative player. I'm your host, CCP Fallout, and together with my wonderful ever-smiling partner, CCP Zymurgist, we hope to showcase some amazing work EVE Online players have created.

Everyone seems to be completely in love with the new Character Creator, which is periodically being tested on the Singularity test server and will make its way to Tranquility with January's release of the EVE Online: Incursion expansion. Even Kotaku is in love with it! Showcased in that article is this gorgeous video by YouTube user sarmatiko, which really shows how detailed you can get when designing your face.



sarmatiko also created a video featuring a female character in the creator back in October. Keep your hands atop your desks, gentlemen!

vexxus5 has created a four race-based videos in a series called "Raise the Flag":

• Gallente: http://www.youtube.com/watch?v=yEmLcoRpCEo
• Amarr: http://www.youtube.com/watch?v=c5-b9TioIto
• Minmatar: http://www.youtube.com/watch?v=ZhDNHZDj8bg
• Caldari: http://www.youtube.com/watch?v=5eAIQR01ZLw

Creativity can express itself in so many different ways visually. Minecraft, which is a game that so many people (including many in the office) are addicted to, has been visited by the Gallente recently. davelantor created this amazing Gallente shuttle in the game, and gives us an inside view on what really makes that bad boy fly.

The EVE Report is a site that I recently added to my feeds list. This community blog features a lot of great news items about our favorite game and players, but also has its very own podcast. What really grabbed my attention however, was a new feature they have on their site. Last time, I mentioned Crazy Kinux's amazing blog pack. The EVE Report does something similar, but with podcasts. For those of you who enjoy aural stimulations, check out their EVE Pod Pack page. A great resource for active podcasts, it also features podcasts of yore and new podcasts.

Speaking of podcasts, Crazy Kinux dropped me a line via Twitter recently to let me know that both he and Maeve Trinity not only have a new podcast, New Eden Chronicles, in the works, but wanted you to know that there will be two additional hosts as well. Congratulations to Memoocan and Ardwulf on their inclusion in the NEC team!

Remember, if you create something that you'd like to share with the EVE Online community, we encourage you to create a post in the My EVE channel. Until next time, fly safely!











More...